Monitoring of skin health

Abstract

The disclosure relates to mobile device allowing for monitoring and / or tracking skin health based on one or more pat-tern light image(s).

Description

[0001] 240674W001

[0002] MONITORING OF SKIN HEALTH

[0003] TECHNICAL FIELD

[0004] The disclosure relates to mobile device allowing for monitoring and / or tracking skin health based on one or more pattern light image(s):

[0005] TECHNICAL BACKGROUND

[0006] Self-improvement or automatized recommendations are rising in the beauty sector. Often this is done by an application embedded in a smartphone. The user regularly does measurements, and the application monitors the result over time.

[0007] SUMMARY

[0008] In an aspect disclosed is a method for monitoring and / or tracking skin health based on one or more pattern light image^):

[0009] triggering an authentication process at least by illuminating at least part of an user provided by at least one infrared pattern illuminator;

[0010] capturing image(s) including one or more pattern light image(s) of at least part of the user;

[0011] providing the captured image(s) to

[0012] at least one authentication process configured to authenticate the user by generating authentication data based on captured image(s) and

[0013] at least one skin detection process configured to determine skin health data by generating one or more skin characteristic(s) associated with a skin of the user based on the captured one or more pattern light image(s);

[0014] providing at least the skin health data for monitoring and / or tracking skin health.

[0015] In another aspect disclosed is an apparatus for monitoring and / or tracking skin health based on one or more pattern light image(s):

[0016] at least one processor configured to trigger an authentication process at least by illuminating at least part of an user provided by at least one infrared pattern illuminator;

[0017] at least one camera configured to capture image(s) including at least one or more pattern light image(s) of at least part of the user;

[0018] at least one interface configured to provide the captured image(s) to at least one processor configured to execute

[0019] at least one authentication process configured to authenticate the user by generating authentication data based on the captured image(s) and240674W001

[0020] 2

[0021] at least one skin detection process configured to determine skin health data by generating one or more skin characteristic(s) associated with a skin of the user based on the captured one or more pattern light image(s);

[0022] at least one output interface and / or display configured to provide at least the skin health data for monitoring and / or tracking skin health.

[0023] In another aspect disclosed is a mobile electronic device comprising at least

[0024] one infrared pattern illuminator configured to illuminate at least part of an user, one or more camera(s) configured to capture images of at least a part of an user illuminated by the at least one infrared pattern illuminator,

[0025] one or more processor(s) configured to monitor and / or track skin health based on one or more pattern light image(s) and

[0026] one or more display(s) and / or output interface(s) configured to display skin health data, the device being configured to:

[0027] trigger, by at least one processor, an authentication process at least by illuminating at least part of an user provided by at least one infrared pattern illuminator;

[0028] capture, by at least one camera, images including at least one or more pattern light image(s) of at least part of the user;

[0029] providing the captured images including the one or more pattern light image(s) to at least one processor configured to execute

[0030] at least one authentication process configured to authenticate the user by generating authentication data based on the captured images and

[0031] at least one skin detection process configured to determine skin health data by generating one or more skin characteristic(s) associated with a skin of the user based on the captured one or more pattern light image(s);

[0032] providing at least the skin health data for monitoring and / or tracking skin health to the at least one display and / or at least one output interface.

[0033] In yet another aspect the present disclosure relates to a computer element with instructions, which when executed on one or more computing node(s) is configured to carry out the steps of the method(s) of the present disclosure or configured to be carried out by the apparatus(es) of the present disclosure.

[0034] Any disclosure, embodiments and examples described herein relate to the methods, the systems, apparatuses, chemical products and computer elements lined out above and below. Advantageously, the benefits provided by any of the embodiments and examples equally apply to all other embodiments and examples.240674W001

[0035] 3

[0036] EMBODIMENTS

[0037] Using electrical and optical sensing-based devices, it is tiresome to remember to regularly do skin measurements in order to get enough data for a close monitoring of skin characteristic(s) . Based on the few data, the results may not be accurate. By providing by one or more mobile electronic device(s) including one or more camera(s), one or more speckle image(s) and / or pattern light image(s) associated with at least a part of one or more object(s), wherein the at least part of the one or more object(s) is being illuminated by coherent electromagnetic radiation associated with a wavelength between 850 nm and 1400 nm and determining based on that skin characteristic(s), users receive reliable and accurate data regarding their skin condition. Moreover, using wearable devices or phones, it is tiresome to remember to regularly do skin measurements in order to get enough data for a close monitoring of skin characteristics. Usually, this can be achieved through push messages sent by an application. But this might be a nuisance for the user and inconvenient. In addition to determine surface roughness data associated with the skin's surface roughness, it is possible to monitor skin characteristics associated with the surface of the skin. This data can be used to reliable assess the skin characteristics and on a regular basis. As users unlock their wearable device and / or phone several times a day, the skin characteristics data can be taken effortlessly, and the data points are dense. Furthermore, skin characteristics as a parameter of skin health, can serve as an additional application to face authentication and the measurements can be at far distances. Therefore, the skin characteristics data associated with the skin's surface roughness can be taken contactless. The electrical and optical sensing-based methods available are mostly contact based. Conductance based methods are found to have low sensitivity for extreme hydration values. Besides the NIR optical sensors being expensive, the presence of other constituents in the skin alters the quantitative measurements. For e.g., moisturizing creams have glycerin or moisturizing ingredients that affects the resultant hydration measurements. This provides only limited regions of measurements, and it is uncomfortable for users to use high-end devices to measure hydration levels of facial skin. Moreover, the high-end devices are not available to a majority of users.

[0038] In the following, embodiments of the present disclosure will be outlined by ways of examples. It is to be understood that the present disclosure is not limited to said embodiments and / or examples.

[0039] In particular for beauty applications it can be tiresome to remember to regularly do measurements in order to get enough data for a close monitoring. Usually this can be achieved through push messages sent by the application. But this might be a nuisance for the user. However, it is inconvenient, an extra app is needed, user must remember the application and must be reminded if necessary. If data is not often generated, the collected data lacks robust-ness / reliability. The present disclosure allows for reliable and close monitoring of skin health by integrating the skin health detection process into the authentication processes the user triggers to authorizes operations on the device.

[0040] Monitoring and / or tracking skin health may include time-dependent measuring of skin health in particular by executing a skin detection process per authentication process triggered by the user of the device. Monitoring and / or tracking skin health may include determining per authentication process the skin health based on the images taken for or during the authentication process. By storing skin health determined during or for previous authentication processes240674W001

[0041] 4

[0042] the skin health may be determined in time-dependent manner. By integrating the skin detection process into the authentication process and storing skin health determined during or for previous authentication processes the skin health may be determined in time-dependent manner based on dense data set with regular measurements taken per authentication process.

[0043] With the proposed face authentication system data is taken with each unlock process. This data can be used to assess the skin health on a regular basis. As users unlock their phone several times a day, the skin health data can be taken effortlessly, and the data points are very dense:

[0044] There are daily multiple unlocks and evaluation of skin condition for a day / week / year in combination with skincare products and evaluation of products. Based on this multiple data acquisition, there are alarms if there is low skin moisture. Furthermore, contextual data can be integrated into the determination of skin health data. Contextual data can relate to location data and / or health data such as temperature, altitude or rppg data. In an example, if the user in addition tracks health data such as the water intake and if it is below a threshold for water supply, the data can be used in addition to the skin characteristics for conclusions on low water supply. In another example the user may track the heart rate based on rppg data, when doing sports. This data can be used in addition to the skin characteristics for conclusions on low water supply. In another example in addition to the skin characteristics a location-bound reminder e.g. aeroplanes, high solar radiation can be taken into account for determining the skin health data. Moreover the recommendation can include this data.

[0045] There may be an alarm if there is low skin moisture. Based on the low skin moisture, there may be conclusions on low water supply.

[0046] Skin characteristic® may be associated with the skin of one or more object(s). Skin characteristic(s) may relate to the characteristic® of the surface of object(s). Skin characteristics may relate to one or more attribute(s) and / or feature of the skin. The attribute(s) and / or feature may define its appearance, texture, color, and / or overall condition. The skin characteristic® may relate to the skin of a human. The skin of a human is the largest organ of the body and may play a crucial role in protecting human(s) from external factors, regulating body temperature, and / or maintaining hydration levels. Determining the different skin characteristics may help in identifying specific skin types, diagnosing skin conditions, and / or developing an appropriate skincare routine. Skin characteristic(s) may relate to static skin characteristic(s). Static skin characteristic® may be constant over a time interval. Static skin characteristic(s) may relate to properties or behavior of skin that do not change over time. Static skin characteristic® may include skin pigmentation and / or skin type.

[0047] Dynamic skin characteristic(s) may change during a time interval. Dynamic skin characteristics may relate to the properties or behavior of the skin that change over time and / or in response to external stimuli. Dynamic skin characteristic® may include skin elasticity, skin hydration, blemishes, sun damage, skin aging, skin pores, skin sensitivity and / or skin texture. There may be skin characteristic® that may relate to both dynamic skin characteristic® and static skin characteristic®.240674W001

[0048] 5

[0049] Skin characteristic(s) may relate to skin type, skin textures, skin pigmentation, skin elasticity, skin hydration, skin sensitivity, skin damages, skin pores and / or skin blemishes.

[0050] Skin type may relate to normal, dry, oily, and combination skin. Skin type(s) may have distinct characteristics and may require different approaches to skincare.

[0051] Skin texture may relate to the smoothness or roughness of the skin's surface. Smoothness may be the inverse of roughness. Skin roughness or smoothness may be determined using skin roughness measures. Skin roughness and / or smoothness may vary based on factors like genetics, age, sun exposure, skin color and / or skincare habits. Skin pigmentation may be determined by the amount of melanin present. Melanin is a pigment that gives color to the skin, hair, and eyes. Skin color may vary greatly across individuals.

[0052] Skin elasticity may refer to its ability to stretch and then return to its original shape. Young, healthy skin may have a good elasticity, while aging or damaged skin may lose its elasticity, leading to wrinkles.

[0053] Skin hydration may relate to a hydration level of the skin. Hydration level of the skin may be crucial for maintaining its health and appearance. Dry skin may lack moisture and may feel tight or flaky, while well-hydrated skin may appear plump and supple.

[0054] Skin sensitivity may relate to skin that reacts easily to external irritants, allergens, or certain skincare products. Sensitive skin may experience redness, itching, or inflammation.

[0055] Skin pores may relate to tiny openings on the surface that allow sweat and sebum (skin oil) to reach the skin's surface. Pore size may vary, and factors like genetics, age, and skincare routine may affect their appearance.

[0056] Blemishes may impact the appearance of the skin. Blemishes may relate to acne, scars, or dark spots, can impact the appearance of the skin. These may be caused by factors like hormonal changes, bacterial infections, or skin injuries.

[0057] Sun damage may relate to overexposure of the skin. Overexposure to the sun's harmful UV rays may lead to various skin issues, including sunburn, premature aging, and / or an increased risk of skin cancer.

[0058] Skin aging may relate to age related changes of the skin. As we age, the skin may undergo various changes, such as the loss of collagen and elastin, which may result in wrinkles, fine lines, and sagging.

[0059] Skin health may relate to a physical and / or physiological condition of the skin. Skin health data may be determined by at least one skin detection process. Skin health may be associated with the skin of a user. Skin health data may relate to one or more skin characteristic(s), which may relate to skin texture, pigmentation, elasticity, hydration, sensitivity, pore size, blemishes, sun damage, and / or age-related changes. Skin health data may relate to one or more skin characteristics and one or more contextual data. Skin health data may further relate to the monitoring and / or tracking of changes in skin characteristics over time or under varying environmental or physiological conditions. In addition to skin characteristics, skin health data may be correlated with contextual data such as location data (e.g., humidity, temperature, UV exposure), and / or health e.g. user habits (e.g., skincare routines, sun protection) and / or physiological states (e.g., hydration level, hormonal fluctuations).

[0060] One or more camera(s) may relate to a device having at least one image sensor configured for generating or recording spatially resolved one-dimensional, two-dimensional or even three-dimensional optical data or information. The240674W001

[0061] 6

[0062] camera may be a digital camera. As an example, the camera may comprise at least one image sensor, such as at least one CCD sensor and / or at least one CMOS sensor configured for recording images.

[0063] The one or more camera(s) may comprise dot camera(s). The one or more camera(s) may comprise an image sensor and a lens. Preferably, the camera may comprise an image sensor, a lens and a polarizer. A lens may relate to an optical element suitable for influencing the expansion of the light beam associated with the coherent electromagnetic radiation. The polarizer may refer to an optical element suitable for selecting the electromagnetic radiation according to its polarization. In particular, the polarizer may refer to an optical element suitable for selecting the coherent electromagnetic radiation according to polarization it. At least part of the electromagnetic radiation, in particular coherent electromagnetic radiation may pass the polarizer while the rest of the electromagnetic radiation, in particular coherent electromagnetic radiation, may be turned away at least partially and / or may be absorbed at least partially. Camera may relate to at least one unit of the optoelectronic apparatus configured for generating at least one image. The image may be generated via a hardware and / or a software interface, which may be considered as the camera. The camera may comprise at least one image sensor, in particular at least one pixelated image sensor. The camera may comprise at least one CMOS sensor and / or at least one CCD chip. For example, the camera may comprise at least one CMOS sensor, which may be sensitive in the infrared spectral range. The camera may have a field of view between 10°x10° and 75°x75°, preferably 55°x65°. The camera may have a resolution below 2 megapixel, preferably between 0.3 megapixel and 1.5 megapixel. Megapixel may relate to a unit for measuring the number of pixels associated with a camera and / or an image. The camera may comprise further elements, such as one or more optical elements, e.g. one or more lenses. As an example, the optical sensor may be a fix-focus camera, having at least one lens which is fixedly adjusted with respect to the camera. Alternatively, the camera may also comprise one or more variable lenses which may be adjusted, automatically or manually. Other cameras, however, are feasible.

[0064] Illumination source may relate to a device suitable for illuminating the object by coherent electromagnetic radiation and / or suitable for emitting coherent electro-magnetic radiation. The illumination source may comprise at least one radiation source. The illumination source may comprise a plurality of radiation sources. The illumination source may comprise for example at least one laser source and / or at least one semiconductor radiation source. A semiconductor radiation source may be for example a light-emitting diode such as an organic, a laser diode and / or inorganic lightemitting diode. Additionally or alternatively, the radiation source may be a VCSEL array and / or a LED. Additionally or alternatively, the illumination source may comprise a VCSEL array and / or a LED. Additionally or alternatively, the illumination source may comprise a VCSEL Flood projector and / or VCSEL dot projector. Further, the illumination source may comprise one or more optical elements. Optical element may be for example a lens, a meta surface element, a diffractive optical element (DOE) or a combination thereof. Hence, an illumination source may comprise one or more radiation sources and one or more optical elements.

[0065] As the coherent electromagnetic radiation associated with a wavelength between 850 nm and 1400 nm may penetrate the skin deeply, a part of the information received from the light reflected from the skin may comprise information independent from the surface roughness which distracts the measurement of the surface roughness. To increase the signal-to-noise ratio a polarizer may be used. The coherent electromagnetic radiation reflected from the240674W001

[0066] 7

[0067] surface of the object may be polarized differently as the light reflected from deeper layers of the human skin. The polarizer may enable a selection of the desired signal from the undesired signal.

[0068] In an embodiment, Coherent electromagnetic radiation may relate to electromagnetic radiation that is able to exhibit interference effects. It may also include partial coherence, i.e. a non-perfect correlation between phase values. Preferably, the coherent electromagnetic radiation may be in the infrared range. The coherent electromagnetic radiation may be associated with a wavelength between 850 nm and 1400 nm. Preferably, the coherent electromagnetic radiation may be associated with a wavelength between 880 nm and 1300 nm. More preferably, the coherent electromagnetic radiation may be associated with a wavelength between 900 nm and 1000 nm and / or wherein the coherent electromagnetic radiation may be associated with a wavelength between 1100 nm and 1200 nm. This may be advantageous since the sunlight has a bandgap in these regions. Followingly, the coherent electromagnetic radiation with the abovementioned wavelengths for generating the speckle image may be differentiated easier from incoming sun light. The use of coherent electromagnetic radiation within the above specified region enables measurements of surface roughness with the presence of sun light such as in the nature. Consequently, measurements of surface roughness may be easily and location-independently used. Overall, an improved signal-to-noise ratio may be achieved and the accuracy of evaluations of surface roughness may be increased.

[0069] Patterned coherent electromagnetic radiation may relate to a plurality of light beams of coherent electromagnetic radiation. Patterned coherent electromagnetic radiation may comprise a plurality of light beams, e.g at least two light beams, preferably at least two light beams. Projection of a light beam of the patterned coherent electromagnetic radiation onto a surface may result in a light spot. Followingly, projecting a plurality of light beams of the patterned coherent electromagnetic radiation onto the object may result in a plurality of light spots on the object(s). The number of light spots may be equal to the number of light beams associated with the patterned coherent electromagnetic radiation. The one or more light spots may be shown in the speckle image. A projection of patterned coherent electromagnetic radiation onto a regular surface may result in a light spot projected onto the regular surface independent of speckle. A projection of patterned coherent electromagnetic radiation onto a regular surface may result in a light spot projected onto the irregular surface comprising at least one speckle, preferably a plurality of speckles. Object may be associated with an at least partially irregular surface. Therefore, the speckle image may comprise a plurality of speckles. If the object may comprise at least partially skin, a plurality of speckle is formed due to the interference of the coherent electromagnetic radiation. Followingly, a light spot may comprise zero, one or more speckle depending on the surface the patterned coherent electromagnetic radiation is projected on. Skin may have an irregular surface. Hence the projection of patterned coherent electromagnetic radiation may result in the formation of speckle within the one or more light spots. A light spot may be a result of the projection of a light beam associated with the patterned coherent electromagnetic radiation. A light spot refers to an arbitrarily shaped spot of coherent electromagnetic radiation. A light spot may refer to a contiguous area illuminated with coherent electromagnetic radiation. Projecting coherent electromagnetic radiation on an irregular surface may result in the formation of speckle. Followingly, the light spot may comprise one or more speckle(s). A light spot may have a diameter between 0.5 mm and 5 cm, preferably 0.6 mm and 4 cm, more preferably, 0.7 mm and 3 cm, most preferably 0.4 and 2 cm.240674W001

[0070] 8

[0071] Patterned coherent electromagnetic radiation may be generated by an illumination source comprising a plurality of light emitters such as a VCSEL array comprises a plurality of VCSELs. An emitter of the plurality of light emitters may emit one light beam. Hence, an emitter of the plurality of light emitters may be associated with the one light spot, with the formation of one light spot and / or with the projection of one light spot. Additionally or alternatively, patterned coherent electromagnetic radiation may be generated by one or more light emitters and an optical element such as a DOE or a metasurface element. A metasurface element may be a meta lense. A meta lense may be at least partially transparent with respect to the coherent electromagnetic radiation and / or may comprise a material associated with a structure on the nanoscale. The optical element may replicate the number of light beams associated with the one or more light emitters and / or may be suitable for replicating the number of light beams associated with the one or more light emitters. For example, the light emitter may be a laser.

[0072] Additionally or alternatively, patterned coherent electromagnetic radiation may be generated by one or more light emitters and an optical element such as a DOE or a metasurface element. The optical element may replicate the number of light beams associated with the one or more light emitters and / or may be suitable for replicating the number of light beams associated with the one or more light emitters. For example, the light emitter may be a laser. An illumination source for generated patterned coherent electromagnetic radiation may be an infrared pattern illuminator. Pattern may relate to an arbitrary known or pre-determined arrangement comprising at least one arbitrarily shaped feature. The pattern may comprise at least one feature such as a point, dot or symbol. The pattern may comprise a plurality of features. The pattern may comprise an arrangement of periodic or non-periodic features. Illumination pattern may refer to a pattern generated and projected by the projector, in particular which used for illuminating the object. The illumination pattern may be an arbitrary pattern comprising a plurality of illumination features.

[0073] A processor specifically may be configured, such as by software programming, for performing one or more evaluation operations. At least one or any component of a computer program con-figured for performing the authentication process may be executed by the processing device. Alternatively, or in addition, the processor may be or may comprise a connection interface. The connection interface may be configured to transfer data from the device to a remote device; or vice versa. At least one or any component of a computer program configured for performing the authentication process may be executed by the remote device. The one or more speckle image(s) and / or pattern light image(s) may be provided to and / or received by a processor. The processor may comprise one or more processors. The processor may determine the surface roughness measure based on the one or more speckle image.

[0074] Speckle image(s) and / or pattern light image(s) may refer to image(s) of an user that may be taken while an object may be illuminated by at least one infrared pattern light illuminator. The light pattern may comprise one or more pattern features. The speckle image(s) and / or pattern light image(s) may include contributions from reflections on the user of the light pattern and / or from the texture of the object. The texture may relate to surface characteristics such as size, shape, density, arrangement, proportion, or the like, of certain parts of the object. Speckle image(s) and / or pattern light image(s) may be speckle image(s) and / or speckle image(s) and / or pattern light image(s).240674W001

[0075] 9

[0076] Speckle may relate to an optical phenomenon caused by interfering coherent electromagnetic radiation due to nonregular or irregular surfaces. Speckles may appear as contrast variations in an image such as a speckle image. Speckle image may refer to an image showing a plurality of speckles. The speckle image may show a plurality of speckles. The speckle image may be generated while the object may be illuminated by coherent electromagnetic radiation associated with a wavelength between 850 nm and 1400 nm. The speckle image may show a speckle pattern. The speckle pattern may specify a distribution of the speckles. The speckle image may indicate the spatial extent of the speckles. The speckle image may be suitable for determining a surface roughness measure. The speckle image may be generated with a camera. For generating the speckle image, the object may be illuminated by the illumination source.

[0077] Surface feature may relate to an arbitrarily shaped structure associated with the surface of the object. In particular, the surface feature may refer to a substructure of the surface associated with the object. A surface may comprise a plurality of surface features. For example, an uplift or a sink may be surface features. Preferably, a surface feature may relate to a part of the surface associated with an angle unequal to 90° against the surface normal.

[0078] Surface roughness may relate to a property of a surface associated with the object. In particular, the surface roughness may characterize lateral and / or vertical extent of surface features. The surface roughness may be evaluated based on the surface roughness measure. The surface roughness measure may quantify the surface roughness. Surface roughness may be a quantitative measure of the irregularities or variations in the surface texture of an object. It provides information about fine-scale details and deviations from ideal smoothness of a surface. Surface roughness may include surface waviness. Surface waviness may include directionality (Anisotropy) of the surface. In aging directionality may be prominent.

[0079] Surface roughness measure(s) may relate to a measure suitable for quantifying the surface roughness. Surface roughness measure may be related to the speckle pattern. Preferably, the surface roughness measure may be suitable for describing the vertical and lateral surface features. Surface roughness measure may comprise a value associated with the surface roughness. Surface roughness measure may refer to a term of a quantity for measuring the surface roughness and / or to the values associated with the quantity for measuring the surface roughness. Surface roughness measure may include at least one of a fractal dimension, speckle size, speckle contrast, speckle modulation, roughness exponent, standard deviation of the height associated with surface features, lateral correlation length, average mean height, root mean square height or a combination thereof. There may be various methods and parameters used to measure surface roughness. A parameter may be the Ra value (arithmetical mean roughness). Ra is calculated as the average of the absolute values of the height deviations from the mean line of the surface profile within a specified sampling length. It represents the average roughness of the surface and is expressed in units of length, such as micrometers (pm) or inches (in).

[0080] Other surface roughness measure(s) may include Rz (maximum height of the roughness profile), Rq (root mean square roughness), Rt (total height of the profile), and Rmax (maximum peak-to-valley height). Each parameter may240674W001

[0081] 10

[0082] provide different information about the surface roughness and can be used depending on the specific requirements of the application.

[0083] Speckle pattern(s) may relate to a distribution of the plurality of speckles. The distribution of the plurality of speckles may relate to a spatial distribution of at least one of the plurality of speckles and / or a spatial distribution of at least two of the plurality of speckles in relation to each other. Spatial distribution of the at least one of the plurality of speckles may relate to a spatial extent of the at least one of the plurality of speckles. Spatial distribution of the at least two of the plurality of speckles may relate to a spatial extent of the first speckle of the at least two speckles in relation to the second speckle of the at least two speckles and / or a distance between the first speckle of the at least two speckles and the second speckle of the at least two speckles.

[0084] A device may comprise the one or more illumination source(s), camera(s) and / or processor(s).

[0085] The device may be a portable device. Portable may refer to the property of at least one object of being moved by human force, such as by a single user. The device may be a static device. The device may be selected from the group comprising: a vehicle, a television device; a game console; a personal computer; a mobile electronic device, particularly a cell phone, and / or a smart phone, and / or a tablet computer, and / or a laptop, and / or a tablet, and / or a virtual reality device, and / or a wearable, such as a smart watch.

[0086] One or more model(s) may be suitable for determining an output based on an input. Model(s) may be configured to determining a surface roughness measure based on the speckle image, preferably based on receiving speckle image.

[0087] Data-driven model(s) may represent a correlation between the surface roughness measure and the speckle image. The data-driven model may obtain the correlation between surface roughness measure(s) and speckle image(s) and / or pattern light image(s) based on a training data set comprising a plurality of speckle image(s) and / or pattern light image(s) and a plurality of surface roughness measures.

[0088] Data-driven model (s) may be parametrized based on a training data set to receive the speckle image and provide the surface roughness measure. The data-driven model may be trained based on a training data set. The training data set may comprise one or more speckle image(s) and / or pattern light image(s) and one or more corresponding surface roughness measure(s). The training data set may comprise a plurality of speckle image(s) and / or pattern light image(s) and a plurality of surface roughness measures. Training the model may include parametrizing the model. The data-driven model may be parametrized and / or trained to provide the surface roughness measure(s) based on the speckle image, in particular receiving the speckle image. Determining the surface roughness measure(s) based on the speckle image may comprise providing the speckle image to a data-driven model and receiving the surface roughness measure from the data-driven model. Providing surface roughness measure based on the speckle image may comprise mapping the speckle image to the surface roughness measure. The data-driven model may be parametrized and / or trained to receive the speckle image. Data-driven model may receive the speckle image at an input layer.240674W001

[0089] 11

[0090] The term training may also be denoted as learning. The term may relate to a process of building the data-driven model, in particular determining and / or updating parameters of the data-driven model. Updating parameters of the data-driven model may also be referred to as retraining. Retraining may be included when referring to training herein. During the training the data-driven model may be adjusted to achieve best fit with the training data, e.g. relating the at least on input value with best fit to the at least one desired output value. For example, if the neural network is a feedforward neural network such as a CNN, a backpropagation-algorithm may be applied for training the neural network, e.g. GLCM CNN. In case of a RNN, a gradient descent algorithm or a backpropagation-through-time algorithm may be employed for training purposes. For example, a neural network may be a classification model e.g. SVM Classifier. Training a data-driven model may include or may refer without limitation to calibrating the model.

[0091] In an embodiment, a model may reflect physical phenomena in mathematical form, e.g., including first-principles models. A physical model may comprise a set of equations that describe an interaction between the object and the coherent electromagnetic radiation thereby resulting in a surface roughness measure. The physical model may be based on at least one of a fractal dimension, speckle size, speckle contrast, speckle modulation, roughness exponent, standard deviation of the height associated with surface features, lateral correlation length, average mean height, root mean square height or a combination thereof. In particular, the physical model may comprise one or more equations relating the speckle image and the surface roughness measure based on equations relating to the fractal dimension, autocorrelation, speckle size, speckle contrast, speckle modulation, roughness exponent, standard deviation of the height associated with surface features, lateral correlation length, average mean height, root mean square height or a combination thereof.

[0092] In an embodiment the disclosure relates to a method for monitoring and / or determining one or more skin characteristics) based on one or more surface roughness measure(s) associated with at least a part of one or more object(s), the method comprising:

[0093] providing by one or more device(s) including one or more camera(s), one or more speckle image(s) and / or pattern light image(s) associated with at least a part of one or more object(s), wherein the at least part of the one or more object(s) is being illuminated by one or more illumination source(s);

[0094] determining one or more surface roughness measure(s) based on the one or more speckle image(s) and / or pattern light image(s);

[0095] determining the one or more skin characteristic(s) associated with a skin to be analyzed of the one or more object(s) based on the one or more surface roughness measure(s) derived based on the one or more speckle image(s) and / or pattern light image(s);

[0096] providing the one or more skin characteristic(s) associated with at least a part of one or more object(s).240674W001

[0097] 12

[0098] In another embodiment the disclosure relates to a system for monitoring and / or determining one or more skin characteristic^) based on one or more surface roughness measure(s) associated with at least a part of one or more object , the method comprising:

[0099] an input providing by one or more device(s) including one or more camera(s), one or more speckle image(s) and / or pattern light image(s) with at least a part of one or more object(s), wherein the at least part of the one or more object(s) is being illuminated by one or more illumination source(s);

[0100] a processor determining one or more surface roughness measure(s) based on the one or more speckle image® and / or pattern light image®;

[0101] a processor for determining the one or more skin characteristic® associated with a skin to be analyzed of the one or more object® based on the one or more surface roughness measure® derived based on the one or more speckle image® and / or pattern light image®;

[0102] an output for providing the one or more skin characteristic® associated with at least a part of one or more object®.

[0103] In another embodiment the disclosure relates to a method for determining one or more skin characteristic® based on monitoring one or more surface roughness measure® associated with at least a part of one or more object®, the method comprising:

[0104] providing by one or more device® including one or more camera®, one or more speckle image® and / or pattern light image® associated with at least a part of one or more object®, wherein the at least part of the one or more object® is being illuminated by one or more illumination source®;

[0105] determining one or more surface roughness measure® based on the one or more speckle image® and / or pattern light image®;

[0106] determining the one or more skin characteristic® associated with a skin to be analyzed of the one or more object® based on the one or more surface roughness measure® derived based on the one or more speckle image® and / or pattern light image®;

[0107] providing the one or more skin characteristic® associated with at least a part of one or more object®.

[0108] In another embodiment, a model may reflect physical phenomena in mathematical form, e.g., including second order statistical measure®. A model may be using Grey-Level Co-occurrence matrix. A GLCM may be used as a feature extractor. A GLCM is a matrix that may be used to describe the relationship between at least two adjacent pixels in a digital image. GLCM is a statistical method that may be used in image processing to analyze texture features of an image.

[0109] In a GLCM, element® may represent the number of times a particular combination of pixel intensities may occur in the image®. The matrix is based on the idea that the relative positions of pixels in an image may provide data about the texture of the image. By analyzing the patterns and relationships between pixel values in an image, the GLCM may be used to extract texture metrics of the image, such as contrast, homogeneity, and entropy.240674W001

[0110] 13

[0111] Dot image(s) may be generated using a dot projector. The speckle pattern may specify a distribution of the speckles. Dot image(s) may include speckles. The speckles may be processed as a whole speckle image with all dots or cropped region of interest with image patch of few spots and / or spots of interest. The image patches with dots containing speckles may also be classified after dimensionality reduction techniques and clustering methods. The feature dimensionality may be reduced into lower dimension for boosting clustering performance for further classification. A statistical model for dot images may be used for unsupervised classification and may use a parametric and / or nonparametric method.

[0112] Object(s) may refer to a physical entity. Object(s) may comprise one or more materials. Materials may include skin, in particular to human's skin, paper, wood, glass, metal oxides such as aluminum oxide, leather or the like. Object may relate to a living organism. Preferably, the object may be a human. Where the object is a living organism such as a human, at least a part of the skin of the living organism such as a human may be illuminated with the coherent electromagnetic radiation. The object may be a human and the surface roughness may be associated with the skin of the human. For this purpose, the speckle image may show at least a part of the skin of the human while being illuminated by coherent electromagnetic radiation. The at least a part of the skin of a human may relate to a part of the face of a human. Surface roughness measure(s) of human skin may be in the range of 15pm to 50pm. Surface roughness measure(s) of the face of human may be less than 15 pm to 30 pm. Surface texture may be less significant in the face of a human compared to other part(s) of the human.

[0113] In an embodiment the one or more speckle image(s) and / or pattern light image(s) associated with at least a part of one or more object(s) may relate to substantially the same part of the one or more object(s). Substantially the same in the current context may include deviation(s), which are due to the imaging process guided by the user. The speckle image may include the respective part of the object(s). The respective part of the object(s) may be in a different area of the image. Nonetheless the respective part of the object(s) may be included in the images. The one or more speckle image(s) and / or pattern light image(s) may relate to one or more cropped speckle image(s) and / or pattern light image(s). The one or more speckle image(s) and / or pattern light image(s) may relate to one or more respective part(s) of the one or more object(s) associated with a skin to be analyzed. The one or more speckle image and / or pattern light image(s) may relate to the same respective part of the object(s) associated with a skin to be analyzed. The one or more speckle image(s) and / or pattern light image(s) may relate to the same respective part of the object(s) associated with a skin to be analyzed for monitoring a difference of the respective part of the object®.

[0114] In an embodiment the one or more speckle image® and / or pattern light image® associated with at least a part of one or more object® may be reduced to a predefined size prior to determining the surface roughness measure®. Reducing the speckle image to a predefined size may be based on applying one or more image augmentation techniques. Reducing the speckle image to a predefined sized may comprise selecting an area of the speckle image of the predefined size and cutting the speckle image to the area of the speckle image of the predefined size. The area of the speckle image of the predefined size may be associated with the living organism such as the human, in240674W001

[0115] 14

[0116] particular with the skin of the living organism. The part of the image other than the area of the speckle image of the predefined size may be associated with background and / or may be independent of the living organism. The part of the image useful for the analysis may be selected. Reducing the size may result in disregarding parts of the speckle image independent of the object or living organism such as a human. The part of the image that may be treated such as moisturized may be selected.

[0117] In an embodiment providing by one or more device(s) including one or more camera(s), one or more speckle image and / or pattern light image(s) associated with at least a part of one or more object(s), wherein the at least part of the one or more object(s) is being illuminated by one or more illumination source(s)may be generated during a face authentication process. In an embodiment one or more flood light image(s) of at least part of the user may be captured, wherein the captured one or more speckle image(s) and / or pattern light image(s) and / or the captured one or more flood light image(s) may be provided to at least one authentication process. The face authentication process may perform at least one face detection using the flood image. Face identification, i.e. assigning an identity to the detected face may be performed. The authentication unit may be configured for identifying the user based on the flood image. Particularly therefore, the authentication unit may forward data to a remote device. Alternatively or in addition, the authentication unit may perform the identification of the user based on the flood image, particularly by running an appropriate computer program having a respective functionality. Identifying may relate to assigning an identity to a detected face and / or at least one identity check and / or verifying an identity of the user.

[0118] The authentication using the flood image may be validated using the surface roughness measure(s). The authentication process may comprise determining if the surface roughness measure corresponds to a surface roughness measure of a human being. For example, the method may comprise determining if the surface roughness measure corresponds to a surface roughness measure of the specific user. Determining if the surface roughness measure corresponds to a surface roughness measure of a human being and / or of the specific user may comprise comparing the surface roughness measure to at least one predefined or predetermined range of values of surface roughness measure, e.g. stored in at least one database e.g. of the device or of a remote database such as of a cloud. In case the determined surface roughness measure is at least within tolerances within the redefined or predetermined range of values of surface roughness measure, the user is authenticated otherwise the authentication is unsuccessful. For example, the surface roughness measure may be a human skin roughness. In case the determined human skin roughness is within the range of 10 m to 150 pm the user may be authenticated. However, other ranges may possible. During the authentication process the speckle image(s) and / or pattern light image(s) and / or surface roughness measure(s) of the object(s) and / or user(s) may be generated and / or stored. The speckle image(s) and / or pattern light image(s) and / or surface roughness measure(s) may be stored and / or used for determining the one or more skin characteristic(s). The speckle image(s) and / or pattern light image(s) and / or surface roughness measure(s) may be provided to an application for determining the one or more skin characteristic(s).

[0119] In an embodiment the authentication process may include facial recognition configured to generate at least one biometric representation of the user and to match the at least one biometric representation with reference template vectors stored for the user.240674W001

[0120] 15

[0121] In an embodiment skin health data may be generated per triggered authentication process based on at least the one or more speckle image(s) and / or pattern light image(s) captured for the authentication process. Skin health data may be generated per triggered unlock event. Generating skin health data per triggered authentication process may relate to generating multiple skin health data point(s). Based on generating skin health data per triggered authentication process, historically skin health data may be generated for previous authentication processes and / or monitoring and / or tracking of time-dependent change in skin health data may be enabled.

[0122] In an embodiment skin health data and the one or more skin characteristic(s) generated per triggered authentication process may be stored for access and / or analysis by the at least one skin detection process.

[0123] Since skin health data and the one or more skin characteristics may be generated per triggered authentication process, multiple skin health data points and multiple skin characteristics may be generated. The generation and storage of this data may facilitate monitoring and / or tracking time-dependent changes in skin health. The stored skin health data and the one or more skin characteristic(s) may be used for determining time-dependent changes in skin health. Processes such as skin detection and / or treatment instructions may thereby be supported.

[0124] In an embodiment the capturing of one or more speckle image(s) and / or pattern light image(s) may be provided to the at least one authentication process and the at least one skin detection process is executed in the same step.

[0125] In an embodiment providing by one or more device(s) including one or more camera(s), one or more speckle image and / or pattern light image(s) associated with at least a part of one or more object(s), wherein the at least part of the one or more object(s) is being illuminated by one or more illumination source(s)may be triggered by a user. The user may initiate the generation of the speckle image based on the mobile electronic device. This is beneficial since many humans own a mobile electronic device such as a smartphone. These devices accompany the human and thus, a measurement of the surface roughness is possible at any time and can be carried out in more natural and less artificial contexts. The speckle image may be initiated by the human operating an application of a mobile electronic device. Therefore, a non-expert user may be enabled to determine surface roughness.

[0126] In an embodiment the one or more speckle image(s) and / or pattern light image(s) associated with at least a part of one or more object(s) may be provided by one or more mobile electronic device(s). A mobile electronic device may relate to portable electronic devices. Mobil electronic device(s) may include smartphones, tablets, wearable devices, and / or laptops.

[0127] In an embodiment the one or more illumination sources may be configured to illuminate an object with patterned coherent electromagnetic radiation associated with a wavelength between 850 nm and 1400 nm.

[0128] In an embodiment the one or more speckle image(s) and / or pattern light image(s) associated with at least a part of one or more object(s) may be generated at a predefined time interval. The predefined time interval may relate to a specific period of time that may have been predetermined or set in advance for a particular purpose. It may be a fixed240674W001

[0129] 16

[0130] duration of time that may be established based on the requirements or needs of a specific task or system. The predefined time interval may be established based on the one or more skin characteristic(s) to be determined. The predefined time interval may be established when determining dynamic skin characteristic(s). The predefined time interval may relate to a short time interval e.g. for determining skin characteristic(s) related to skin aging or the like. The predefined time interval may relate to a short time interval e.g. for determining skin characteristic(s) related to skin hydration or the like.

[0131] In an embodiment the one or more speckle image(s) and / or pattern light image(s) associated with at least a part of one or more object(s) may be generated before and / or after one or more treatment(s) of at least a part of one or more object(s). Skin treatment(s) may relate to an exposure of the skin. Skin treatment(s) may relate to various methods and / or procedures used to improve the characteristic(s), appearance(s) and / or function(s) of skin. Skin treat-ment(s) may be based on the skin ch aracteristic(s) . Moisturizing may be an example of skin treatment(s), in order to hydrate the skin, increase smoothness and / or prevent dryness. Applying makeup may be another example of skin treatment(s).

[0132] In an embodiment skin health data may be associated with at least one time-dependent change in skin health and / or at least one treatment instruction. Time-dependent change may relate to one or more time dependent behavior(s). Time dependent change may relate to variation in skin health that may occur over a specified period such as between two or more points in time. Time-dependent change may relate to the change in skin health based on skin characteristic(s) and / or contextual data. Treatment instruction may relate to recommendation(s).

[0133] Treatment instruction may refer to a set of recommendation (s). Treatment instruction may comprise suggested actions based on the determined skin health data. Treatment instruction(s) may include action(s) such as moisturizing, applying makeup, change of location or other procedures designed to enhance skin health. Treatment instructions may be based on time-dependent changes.

[0134] In an embodiment determining the one or more skin characteristic(s) associated with a skin to be analyzed of the one or more object(s) based on one or more surface roughness measure(s) of the one or more speckle image(s) and / or pattern light image(s) may include determining one or more time dependent behavior(s) based on of two or more surface roughness measures determined for different point(s) in time. Determining one or more time dependent behaviors) based on of two or more surface roughness measures determined for different point(s) in time may include determining one or more difference(s) of two or more surface roughness measures. Time dependent behavior(s) may relate to difference(s) in two or more speckle image(s) and / or pattern light image(s) at two different points in time. Difference(s) may be determined by performing a subtraction of two or more surface roughness measure(s) of the two or more speckle image(s) and / or pattern light image(s). The two or more speckle image(s) and / or pattern light image(s) may relate to the same at least part of the object(s). The differences may be determined by determining an average surface roughness of the at least part of the object(s) and performing a subtraction. The differences may be determined by defining a range or a threshold for the parameter(s) of the skin characteristic(s).240674W001

[0135] 17

[0136] In an embodiment determining the one or more skin characteristic(s) associated with a skin to be analyzed of the one or more object(s) based on one or more surface roughness measure(s) of the one or more speckle image(s) and / or pattern light image(s) may include determining one or more intensity changes of one or more laser speckles of two or more speckle image(s) and / or pattern light image(s).

[0137] In an embodiment determining the one or more skin characteristic(s) associated with a skin to be analyzed of the one or more object(s) based on one or more surface roughness measure(s) of the one or more speckle image(s) and / or pattern light image(s) may include generating one or more speckle image(s) and / or pattern light image(s) from one or more angle(s). The projector(s) may be projected at a certain angle to the at least part of the one or more object(s). Based on the changes of the angle(s), a scattering may change. If the face may be imaged at a different angle the angle(s) may be constant, but the face may be moved. Specular reflection (s) and / or diffuse reflection (s) may be dependent on the one or more angle(s).

[0138] In an embodiment the skin detection process may include determining one or more surface roughness measure(s) based on the captured one or more speckle image(s) and / or pattern light image(s) and to generate the one or more skin characteristic(s) based on the one or more surface roughness measure(s).

[0139] In an embodiment determining the one or more surface roughness measure(s) based on the one or more speckle image(s) and / or pattern light image(s) includes determining the one or more surface roughness measure(s) based on a speckle pattern in the speckle image. Determining the one or more surface roughness measure(s) based on the speckle image(s) and / or pattern light image(s) may relate to determining the surface roughness measure based on a speckle pattern. Determining the surface roughness based on the speckle pattern may refer to determining the surface roughness based on a distribution of a plurality of speckles in the speckle image. Determining the surface roughness measure based on the distribution of the plurality of speckles in the speckle image may refer to determining the distribution of the plurality of speckles in the speckle image. Determining the distribution of the speckles may comprise determining at least one of fractal dimension associated with the speckle image, speckle size associated with the speckle image, speckle contrast associated with the speckle image, speckle modulation associated with the speckle image, roughness exponent associated with the speckle image, standard deviation of the height associated with surface features associated with the speckle image, lateral correlation length associated with the speckle image, average mean height associated with the speckle image, root mean square height associated with the speckle image or a combination thereof. Additionally or alternatively, determining the surface roughness measure may comprise determining at least one of fractal dimension associated with the speckle image, speckle size associated with the speckle image, speckle contrast associated with the speckle image, speckle modulation associated with the speckle image, roughness exponent associated with the speckle image, standard deviation of the height associated with surface features associated with the speckle image, lateral correlation length associated with the speckle image, average mean height associated with the speckle image, root mean square height associated with the speckle image or a combination thereof.240674W001

[0140] 18

[0141] As the speckles are caused by the irregularities of the surface, the speckles reflect the roughness of the surface. Determining the surface roughness measure based on the speckles in the speckle image may utilize the relation between the speckle distribution and the surface roughness.

[0142] In an embodiment determining the one or more surface roughness measure(s) based on the one or more speckle image(s) and / or pattern light image(s) may include providing the one or more speckle image(s) and / or pattern light image(s) to a model configured to determine the surface roughness measure based on the one or more speckle image and / or pattern light image(s). The model may be a data-driven model and may be parametrized and / or trained based on a training data set comprising a plurality of speckle image(s) and / or pattern light image(s) and corresponding surface roughness measures or indications of surface roughness measures. Additionally or alternatively, the model may be a physical model.

[0143] In an embodiment determining the one or more surface roughness measure(s) based on the one or more speckle image(s) and / or pattern light image(s) includes using one or more image texture analysis model(s). The image texture analysis model(s) may include one or more GLCM model(s). GLCM model(s) may include Quantization, employing a Co-occurence matrix, symmetrizing the matrix and normalizing the matrix. A gray-level co-occurrence matrix (GLCM) is a matrix that may be used to describe the relationship between two adjacent pixels in digital image(s). GLCM is a statistical method that may be used in image processing to analyze texture features of an image. GLCM is a known concept and for example described in Youssef D., El-Ghandoor H., Kandel H., El-Azab J., Hassab-Elnaby S. Estimation of Articular Cartilage Surface Roughness Using Gray-Level Co-Occurrence Matrix of Laser Speckle Image. Materials. 2017; 10:714. doi: 10.3390 / ma10070714.

[0144] GLCM may be used to characterizes texture(s) of an image. In the context of speckle analytics GLCM may be used to perform a computation of speckle pattern texture. GLCM may use second order statistical measure for features that may be used to infer the degree of correlation between pairs of pixels. GLCM may be used to determine how often two-pixel values are separated by a certain distance d and lie along a certain direction 0 may be related. In other words, GLCM may be used to determine spatial relationship around neighborhood pixels with a predefined pixel offset (d). Pixel offset (d) may describe a distance between neighborhood and reference pixel. Direction of offset (0) may be used to describe an angle between the pixels. Results may describe a joint probability distribution. In a GLCM, each element may represent the number of times a particular combination of pixel intensities occurs in the image.

[0145] The output of the GLCM model may be the normalized GLCM for each window. The moving window may slide over the speckle image and the texture metric(s) and / or based on that, texture maps may be generated.

[0146] In an embodiment determining the one or more surface roughness measure(s) based on the one or more speckle image(s) and / or pattern light image(s) includes determining one or more texture metric(s) and / or one or more texture map(s) associated with the one or more speckle image(s) and / or pattern light image(s). Texture metric(s) are quantitative measurements or parameters that may be used to characterize and describe the texture of image(s) or surface®. Texture metrics may provide numerical values that represent aspects of texture, such as smoothness, roughness, regularity, or complexity. The one or more texture metrics may include features such as contrast, dissimilarity,240674W001

[0147] 19

[0148] homogeneity, entropy, variance, mean, correlation or the like. The texture map(s) may be created by mapping the texture metric(s) to the speckle image(s) and / or speckle image(s) and / or pattern light image(s). The complete speckle image(s) and / or speckle image(s) and / or pattern light image(s) and / or a predefined part of speckle image(s) and / or pattern light image(s) may be recreated based on the texture metrics and / or textures maps of the moving window.

[0149] In an embodiment the one or more skin characteristic(s) include one or more parameter(s) selected from the group consisting of: one or more parameter(s) associated with static skin characteristic(s) and / or one or more parameters associated with dynamic skin characteristic(s).

[0150] For determining dynamic skin characteristic(s) two or more speckle image(s) and / or pattern light image(s) may be provided. The two or more speckle image(s) and / or pattern light image(s) may be provided at a predefined time interval and / or before and after a treatment such as applying a moisturizer.

[0151] In an embodiment the one or more skin characteristic(s) associated with the one or more object(s) include one or more skin type derivation(s). Based on the determined surface roughness, one or more skin type(s) may be derived. For determining the skin type(s) an average skin roughness measure of at least part of one or more object(s) may be determined. The average skin roughness measure before a treatment of the at least part of the one or more object(s) may be determined. A second speckle image of the same at least part of the one or more object(s) may be generated. For determining the skin type(s) an average surface roughness measure of at least part of one or more object may be determined.

[0152] In an embodiment the surface roughness measure(s) are associated with at least a part of skin of one or more human®. The speckle image® and / or pattern light image® may show the skin of a living organism, in particular the skin of the living organism such as the skin of the human.

[0153] In an embodiment the authentication process may be triggered by at least one unlock event related to the user. Unlock event may relate to the user requesting access to one or more functions associated with a device. Based on the authentication process the user may be allowed or declined to access one or more functions associated with a device.

[0154] In an embodiment the skin detection process may include determining the level of roughness associated with the skin or an area of skin of the user by analyzing the captured one or more speckle image® and / or pattern light image® with respect to textural features of the skin's outermost epidermal layer and generating one or more skin characteristic® related to a smoothness of the skin or an area of skin of the user. Analyzing textural features of the skin's outermost epidermal layer may relate to determining one or more texture metric® and / or one or more texture map® based on the one or more speckle image® and / or speckle image® and / or pattern light image®.240674W001

[0155] 20

[0156] In an embodiment the skin detection process may include determining skin health data by detecting at least one region of interest in the captured one or more speckle image(s) and / or pattern light image(s) and generating one or more skin characteristic(s) for the at least one region of interest associated with at least one area of the user's skin. Region of interest may comprise a preselected and / or predefined region of the pattern image light image(s) and / or speckle image(s) and / or speckle image(s) and / or pattern light image(s). The region of interest may relate to the whole image or one or more parts of the image that may be disjoint or may overlap. Region of interest may be preselected and / or predefined by features being comprised in the pattern image light image(s) and / or speckle image(s) and / or speckle image(s) and / or pattern light image(s). For example, region of interest may be at least a part of a user's face. Region of interest may be selected based on the features of a user's face.

[0157] In an embodiment the skin detection process may include determining skin health data based on contextual data. Contextual data may relate to data derived from sensors, which may relate to location data and / or health data that may be used to determine skin health data in addition to skin characteristics. The skin detection process may include determining skin health by evaluating surface roughness measurements in combination with contextual data. Contextual data may be continuously or intermittently derived by one or more sensors and made available for skin health determination e.g. through an operating system of the device.

[0158] In an embodiment contextual data may comprise location data associated with the location of the user and / or health data associated with the user.

[0159] Location data associated with the current location of the user may relate to the user's position when the authentication process may be triggered and / or when the one or more pattern light images of at least part of the user may be captured. Location data may relate to data associated with a characteristic of the surrounding of the person, for example a physical or chemical characteristic of the surrounding of the person. The characteristic of the surrounding of the person may have an influence on the skin health data of the person. Generating location data associated with the current location of the user may be triggered by the authentication process and / or by capturing the one or more pattern light images of at least art of the user. Generating location data associated with the current location of the user may be done with every unlock in addition to authentication. Location data associated with the current location of the user may relate to data having an implicit or explicit association with a location relative to earth. Location data associated with the current location of the user may relate to data indicating a position. Location data associated with the current location of the user may relate to a geolocation. Geolocation may relate to the process of determining the geographic location of a mobile device using various technologies. Location data may comprise sensor data. Location data may comprise the location of the person, for example the geolocation such as the GPC coordinates, the height above see level, distance to a reference point such as the device, acceleration, orientation with regard to gravity; weather conditions such as air temperature, air pressure, air humidity, wind speed, wind direction, ambient light intensity; time or date; air pollutant levels like CO2 concentration, CO concentration, ozone concentration, nitrogen oxide concentration, sulfur dioxide concentration, fine dust concentration, volatile organic compounds level. Location data may include time, date and / or metadata such as device or sensor identifiers.240674W001

[0160] 21

[0161] Location data may be acquired based on Global Positioning System (GPS), which may use satellite signals to pinpoint the device's location. Location data may be acquired based on Cellular Network Triangulation, which may use signals from nearby cell towers to estimate the device's location. Location data may be acquired based on Wi-Fi Positioning, which may identify nearby Wi-Fi networks and use their signal strength to estimate location data. Location data may be acquired based on Location Services. Mobile devices may have built-in location services that may enable applications to access location data.

[0162] Location data may be recorded by a sensor capable of determining the location data. The sensor may be integrated into the device such as a portable device. The sensor may be communicatively coupled to device, for example via a wireless communication or via internet. Examples for sensors may be a GPC receiver, an accelerometer, a gyroscope, an altimeter, a goniometer, a distance sensor like a time-of-flight sensor, a barometer, a light sensor, a thermometer, a gas sensor and / or an altitude sensor.

[0163] Location data may be received from a data storage medium. The data storage medium may be part of the device, or it may be a remote storage device, for example a computer system or a cloud system.

[0164] Location data may be received from a database, for example from a database on a remote storage system, in response to a request containing time and / or geographic location. A remote storage system may refer to a system which is far from the person of the measurement, for example a cloud server or a database server. For example, a request containing the GPS coordinates of the person and the time of skin detection process may be sent to a cloud server having a weather database. The cloud server may in response to the request send weather data corresponding to the time and location of the request.

[0165] Health data associated with the user may relate to data pertaining to the physiological, biological, behavioral, or clinical state of the user. Acquiring and / or generating health data associated with the user may be triggered by an authentication process and / or by capturing one or more pattern light images of at least part of the user, by user interaction, and / or by continuous or intermittent sensor operation. Health data associated with the user may relate to data indicating a physiological measurement or clinical parameter.

[0166] Health data may comprise sensor data. Health data may comprise measurements or observations of the user's physiological state, for example, heart rate, heart-rate variability, respiration rate, blood oxygen saturation, skin temperature, activity level, sleep duration, or movement patterns; clinical information such as medication adherence, symptom reports, or self-assessments; behavioral data such as activity, nutrition, water intake and / or screen time. Health data may include time, date and / or metadata such as device or sensor identifiers. Health data may be acquired based on sensors and / or user- entered data.

[0167] Health data may be acquired by a sensor capable of determining health data. The sensor may be integrated into the device such as a smartphone, wearable, or portable device. The sensor may be communicatively coupled to the device, for example via wireless communication or via the internet. Examples of sensors may include an optical sensor, photoplethysmography (PPG) sensor, electrocardiogram (ECG) sensor, accelerometer, gyroscope, microphone, thermometer, or other physiological or environmental sensors. Health data may be received from a data storage medium. The data storage medium may be part of the device, or it may be a remote storage device, for example a240674W001

[0168] 22

[0169] computer system or a cloud system. Health data may be received from a database, for example from a database on a remote storage system, in response to a request. A remote storage system may refer to a system which is far from the user, for example a cloud server or a database server. For example, a request containing the user's identifier, time and / or additional data may be sent to a cloud server having a health data database. The cloud server may in response to the request send health data corresponding to the time, user, and / or additional data.

[0170] In an embodiment the skin detection process may include determining skin health data based on historically generated skin characteristic(s) determined from one or more speckle image(s) and / or pattern light image(s) captured during previous authentication processes, wherein the skin detection process includes determining skin health data based on location data associated with the current location of the user. Historically generated skin characteristic(s) may refer to skin characteristic(s) that may have been previously generated and stored from earlier authentication processes. The historically generated skin characteristics may be generated from earlier skin detection processes by speckle image(s) and / or pattern light image(s) that may have been taken in the past, generating data points over time.

[0171] In an embodiment skin health data may relate to time-dependent changes in skin health, wherein a notification to the user is provided, if a change in skin health is detected by the skin detection process. The notification may be displayed on a smart device such as a smartphone, wearables or other connected electronics. The notification may aim to inform the user. Notification may include an alarm such as an audible alarm, visual alert and / or a text message indicating the change in skin health. Notification may include further instructions and / or recommendations.

[0172] In an embodiment skin health data may relate to time-dependent change in skin health and / or at least one treatment instruction, wherein the at least one treatment instruction is generated based on the time-dependent change in skin health, wherein the at least one treatment instruction is provided, if a change in skin health is detected by the skin detection process.

[0173] In an embodiment skin health data may relate to time-dependent change in skin health and / or at least one treatment instruction, wherein at least one treatment instruction is generated based on the contextual data. Contextual data may relate to location data and / or health data. For determining skin health data, a time-dependent change in skin health may be generated based on skin characteristic(s) and / or contextual data and previously generated skin health data which may be generated based on previously determined skin characteristic(s) and / or contextual data. A change in the contextual data may relate to a change in the generated skin health data. A change in the contextual data may relate to a weather change and / or a change in the sleep data that may be taken into account of the skin detection process. The treatment instruction may be generated based on that change. In an example, a user in addition to the generation of skin health data triggered by an authentication process, may monitor health data using a wearable device. Over several days, the skin detection process may determine that the user's skin hydration level has decreased. At the same time, contextual data may show that the user has recently traveled to a location with colder, drier weather and has recorded shorter sleep durations via a wearable. Based on these changes in both240674W001

[0174] 23

[0175] weather and sleep data, a time-dependent decrease in skin hydration may be validated. As a result, a treatment instruction advising the user to apply moisturizer more frequently and to ensure adequate sleep to help improve skin health.

[0176] In an embodiment skin health data may be associated with skin hydration and the at least one treatment instruction is associated with skin hydration. Skin hydration may relate to a hydration level of the skin. Hydration level of the skin may be crucial for maintaining its health and appearance. Dry skin may lack moisture and may feel tight or flaky, while well-hydrated skin may appear plump and supple. A treatment instruction may relate to the application of hydration such as moisturizer which may be applied to the skin.

[0177] BRIEF DESCRIPTION OF THE DRAWINGS

[0178] In the following, the present disclosure is further described with reference to the enclosed figures. The same reference numbers in the drawings and this disclosure are intended to refer to the same or like elements, components, and / or parts.

[0179] Fig. 1 illustrates an example system for measuring a surface roughness.

[0180] Fig. 2 illustrates an example of a surface associated with an object.

[0181] Fig. 3 illustrates an example of a Gray Level Co-occurrence Matrix (GLCM).

[0182] Fig. 4 illustrates an example model for generating texture maps from one or more speckle image(s) and / or pattern light image(s) using a GLCM model.

[0183] Fig. 4a illustrate examples of normalized GLCM matrix images for different roughness parameters of silicon.

[0184] Fig. 5 illustrates an example of a process for determining a skin smoothness score and derive a skin type based on surface roughness measures.

[0185] Fig. 6 illustrates an example of a method for monitoring one or more skin characteristic(s) based on one or more surface roughness measure(s) associated with at least a part of one or more object(s).

[0186] Fig. 7 illustrates an example of a process for generating skin health data and generation authentication data.

[0187] Fig. 8 illustrates an example of a device and sensors for generating skin health data.

[0188] The following embodiments are mere examples for implementing the method, the system or application device disclosed herein and shall not be considered limiting.240674W001

[0189] 24

[0190] DETAILED DESCRIPTION

[0191] Fig. 1 illustrates an example system for measuring a surface roughness. The system may comprise an Illumination source 104, a camera 106 and a processor 108. The surface roughness may be determined with respect to an object. Preferably, the object may be a living organism 114 such as a human. The living organism 114 may have skin. The skin may be associated with a surface roughness. The surface roughness may be evaluated based on a surface roughness measure. The skin may be exposed to coherent electromagnetic radiation associated with a wavelength between 850 nm and 1400 nm.

[0192] The coherent electromagnetic radiation associated with a wavelength between 850 nm and 1400 nm may be emitted by the illumination source 104. The illumination source 104 may comprise one or more radiation sources such as a VCSEL array or a single laser diode. The radiation source may be associated with one or more light beams. For example, a single laser diode may emit one light beam, whereas a VCSEL array may emit a plurality of light beams. Preferably the number of light beams correspond to the number of VCSELs in the VCSEL array. Thus, the illumination source may emit a plurality of light beams. The plurality of light beams may result in projecting a pattern onto the object. The illumination source 104 may emit patterned coherent electromagnetic radiation. Patterned coherent electromagnetic radiation may be suitable for projection of a pattern onto the object. The illumination source 104 may comprise one or more optical elements. An optical element may be suitable for splitting and / or multiplication of light beams. Examples for optical elements may be diffractive optical elements, refractive optical elements, meta surface elements, lenses or the like. Illumination source 104 comprising a single laser diode or a VCSEL array in combination with an optical element may result in illuminating the object with patterned coherent electromagnetic radiation. The illumination source 104 may be associated with a field of illumination as indicated by the two lines originating from the illumination source 104.

[0193] Depending on the body part illuminated by the coherent electromagnetic radiation, different skin surface roughness measures may be determined. Different body parts of the living organism 114 may be associated with different skin roughness 116. For example, a hand may be associated with a higher skin roughness whereas the face may be associated with a lower skin surface roughness. The surface roughness may be characteristic for a body part of the living organism 114 and / or for the identity of the living organism 114. The surface roughness measure of the living organism 114 and / or the respective body part of the loving organism 114 may change over time. The changing of the surface roughness measure may be based on the hydration of the skin. The application of hydration may cause changes in the outermost layer of the skin, stratum corneum and thereby may change its structural micro-topology with the reduction in the depth and the width. Additionally, the layer may swell up and the roughness reduces. The changes in roughness of the skin may be correlated with the presence of hydration on different types of the skin (dry, normal and oily).

[0194] One or more speckle image(s) and / or pattern light image(s) may be generated while the living organism 114 may be illuminated with coherent electromagnetic radiation, preferably patterned coherent electromagnetic radiation. Coherent electromagnetic radiation may interact with the surface of the living organism 114 once it may be projected onto the skin of the living organism 114. Coherent electromagnetic radiation may form speckle when interacting with a non-homogeneous and uneven surface such as skin. The different wavefronts of the coherent electromagnetic240674W001

[0195] 25

[0196] radiation may interact by means of interference. The interference of the different wavefronts of the coherent electromagnetic radiation may result in contrast variations of the coherent electromagnetic radiation on the skin of the living organism 114. The contrast variations may depend on the surface roughness associated with the surface which the coherent electromagnetic radiation is illuminating. The surface roughness associated with the skin may influence the formation of speckles such as the size and orientation of the speckle. Followingly, analysis of the speckle may result in a surface roughness measure.

[0197] For analyzing the speckle, one or more speckle image(s) and / or pattern light image(s) may be generated with a camera 106. The camera 106 may comprise a sensor 110. The camera 106 may comprise a lens 112. Based on the lens 112, the one or more speckle image(s) and / or pattern light image(s) may comprise the speckle formation in imaging space and may show a subjective speckle pattern. In another example, the camera 106 may comprise a polarizer. The coherent electromagnetic radiation may be in the infrared range. Thus, the coherent electromagnetic radiation may penetrate deeper than the epidermis. The surface roughness measure may specify the surface roughness associated with the surface of skin. Data, which may be obtained by coherent electromagnetic radiation penetrating for example the dermis or deeper may overlay the desired information relating to the surface of the skin.

[0198] Fig. 2 illustrates an example of the interaction of the coherent electromagnetic radiation with the surface of the skin. A polarizer may be suitable for selecting the coherent electromagnetic radiation reflected from the surface of the skin and may be suitable for de-selecting parts of the coherent electromagnetic radiation having interacted with skin layer(s) such as the dermis or deeper layers.

[0199] Camera 106 may be associated with a field of view as indicated by the two lines originating from the camera 106. The camera 106 may have a field of view between 10°x10° and 75°x75°, preferably 55°x65°. The camera 106 may have a resolution below 2 megapixels, preferably between 0.3 megapixel and 1.5 megapixel. Examples for the one or more speckle image(s) and / or pattern light image(s) can be found in Fig. 4.

[0200] The field of illumination may correspond at least partially to the field of view. At least a fraction of the field of view associated with the camera 106 may be independent of illumination with coherent electromagnetic radiation. Followingly, the one or more speckle image may show at least in parts the object under illumination with coherent electromagnetic radiation. The camera 106 may operate in a far field. In contrast to a microscope like or in other words near field observation, the image(s) of the far field observation may show speckles as features.

[0201] The one or more speckle image(s) and / or pattern light image(s) may be provided to and / or received by a processor 108. The processor 108 may comprise one or more processors. The processor 108 may determine the surface roughness measure based on the one or more speckle image. The processor 108 may determine the surface roughness measure as described within the context of Fig. 4. Processor may be an arbitrary logic circuitry configured to perform basic operations of a computer or system, and / or, generally, to a device which is configured for performing calculations or logic operations. In particular, the processor, or computer processor may be configured for processing basic instructions that drive the computer or system. It may be a semiconductor-based processor, a quantum processor, or any other type of processor configures for processing instructions. As an example, the processor may be or may comprise a Central Processing Unit ("CPU"). The processor may be a ("GPU”) graphics processing unit, (“TPU”) tensor processing unit, ("CISC") Complex Instruction Set Computing microprocessor, Reduced Instruction Set Computing ("RISC") microprocessor, Very Long Instruction Word ("VLIW") microprocessor, or a processor implementing240674W001

[0202] 26

[0203] other instruction sets or processors implementing a combination of instruction sets. The processing means may also be one or more special-purpose processing devices such as an Application-Specific Integrated Circuit ("ASIC"), a Field Programmable Gate Array ("FPGA"), a Complex Programmable Logic Device ("CPLD"), a Digital Signal Processor ("DSP"), a network processor, or the like. The methods, systems and devices described herein may be implemented as soft-ware in a DSP, in a micro-controller, or in any other side-processor or as hardware circuit within an ASIC, CPLD, or FPGA. It is to be understood that the term processor may also refer to one or more processing devices, such as a distributed system of processing devices located across multiple computer systems (e.g., cloud computing), and is not limited to a single device unless otherwise specified. The processor may also be an interface to a remote computer system such as a cloud service. The processor may include or may be a secure enclave processor (SEP). An SEP may be a secure circuit configured for processing the spectra. A "secure circuit" is a circuit that protects an isolated, internal resource from being directly accessed by an external circuit. The processor may be an image signal processor (ISP) and may include circuitry suitable for processing images, in particular images with personal and / or confidential information.

[0204] The one or more system component(s) may be part of a device, e.g. a device for measuring a surface roughness. In other embodiments, the system components may be separated between a plurality of devices. For example, the processor 108 may be a server, whereas the illumination source 104 and the camera 106 may be part of one device such as a mobile electronic device. The camera 106 may provide the one or more speckle image(s) and / or pattern light image(s) to the processor 108. The processor 108 may provide the one or more surface roughness measure(s) to a device for displaying the surface roughness measure and / or a device for processing the surface roughness measure. In an example, the device comprising the camera 106 and the illumination source 104 may further comprise a display for displaying the one or more surface roughness measure(s) and / or a surface roughness processor configured for processing the one or more surface roughness measure(s) further. Optionally, the device may comprise the processor 108.

[0205] Fig. 2 illustrates an example of a surface associated with an object.

[0206] The object may be a living organism as described in the context of Fig. 1. Stratum corneum (SC) may be the outermost layer of the epidermis. Corneocytes cells within SC layer may form a lipid filled intracellular matrix. The Lipids in SC may maintain a skin barrier and may control Trans-Epidermal-Water-Loss (TEWL). For example, for normal skin, TEWL is low. For dry skin, water delivered from the Epidermis may be preserved and / or water from skin surface may be evaporated. Based on the SC layer, a classification of skin type depending on moisture content may be enabled. Dry skin may have a SC with lower water content (<10%). The skin may have a dull and rough appearance. Dry skin may lack sebum and lipids to retain moisture. Dry skin may have an increased TEWL value. Oily skin may have more sebum secretion, visible pores and a shiny texture. Normal skin may have a SC with water content of 10%. Normal skin may have a well-balanced sebum and moisture content.

[0207] The surface of the living organism may comprise a plurality of surface features. Surface features may be lateral surface features and / or vertical surface features. Surface features may occur in the stratum corneum. The lateral surface features may be quantified according to a length of a sink on the surface. The vertical surface features may be quantified according to a height of an uplift on the surface. This surface may be illuminated by coherent240674W001

[0208] 27

[0209] electromagnetic radiation emitted from the illumination source as described in the context of Fig. 1. One or more speckle image(s) and / or pattern light image(s) may be generated while the surface may be illuminated by coherent electromagnetic radiation with the camera as described in the context of Fig. 1. Laser speckles may be results of random interference effects consisting of a dark and bright granular pattern.

[0210] Fig. 3 illustrates an example of a Gray Level Co-occurrence Matrix (GLCM).

[0211] A gray-level co-occurrence matrix (GLCM) is a matrix that may be used to describe the relationship between two adjacent pixels in digital image(s). GLCM is a statistical method that may be used in image processing to analyze texture features of an image. GLCM is a known concept and for example described in Youssef D., El-Ghandoor H. , Kan-del H., El-Azab J., Hassab-Elnaby S. Estimation of Articular Cartilage Surface Roughness Using Gray-Level Co-Occurrence Matrix of Laser Speckle Image. Materials. 2017; 10:714. doi: 10.3390 / ma10070714.

[0212] GLCM may be used to characterizes texture(s) of an image. In the context of speckle analytics GLCM may be used to perform a computation of speckle pattern texture. GLCM may use second order statistical measure for features that may be used to infer the degree of correlation between pairs of pixels. GLCM may be used to determine how often two-pixel values are separated by a certain distance d and lie along a certain direction 0 may be related. In other words, GLCM may be used to determine spatial relationship around neighborhood pixels with a predefined pixel offset (d). Pixel offset (d) may describe a distance between neighborhood and reference pixel. E.g. d=1 infers correlation between neighboring pixels, d=2 means pixel pairs are separated by one pixel. Direction of offset (0) may be used to describe an angle between the pixels, e.g. as shown in Fig. 3: 0°,45°, 90° or 135°. Results may describe a joint probability distribution.

[0213] In a GLCM, each element may represent the number of times a particular combination of pixel intensities occurs in the image.

[0214] In an example the speckle image has size M x M pixels and gray-level values L = {0, 1, 2,...,G-1} with horizontal and vertical dimensions of M ={1, 2,...,m} and N = {1, 2,...., n}, respectively. Then, each elementof a G x G GLCM (CM) is the number of times that any two pixels (x, y) and (x + d, y + d) with gray-level values of F(x, y) = I and F(x + d, y + d) = j, respectively, separated by a distance d along a direction of 0={O°, 45°, 90° or 135°} are repeated:

[0215] A f 1, if F(x,v) - i and Ffx + d, y + d) - j

[0216]

[0217] > ) ) •; " '

[0218] Q otherwise

[0219] x = 1 y = 1 t '

[0220] GLCM may be symmetric since the calculation is maintained in the forward and backward directions. The elements of the GLCM may be normalized thus the sum of the elements of the GLCM is equal to one and to give the probability of GLCM:

[0221] <

[0222]

[0223] where I and j are the horizontal and vertical spatial dimensions of CM, G is the maximum gray-level value of the image and P is the normalized GLCM.240674W001

[0224] 28

[0225] The matrix is based on the idea that the relative positions of pixels in an image may provide data about the texture of the image. By analyzing the patterns and relationships between pixel values in an image, the GLCM may be used to extract texture features of the image, such as described in Detail in Fig. 4.

[0226] Fig. 4 illustrates an example model for generating texture maps from one or more speckle image(s) and / or pattern light image(s) using a GLCM model.

[0227] One or more speckle image(s) and / or pattern light image(s) may be generated as described in the context of Figure 1. The image(s) may have a certain height and widths (pixels). The pixel(s) may have different gray scale values. A moving window, e.g. a 3x3 moving window may be moved on the speckle image. The 3x3 window may have in each a predefined number of pixels. The 3x3 window may move to the right with a certain stride value till it parses the complete width. Then the window may move on, it may hop down to the beginning (left) of the image with the same stride value and may repeat the process until the entire image is traversed.

[0228] Based on the pixels of the window the GLCM model may be applied. The GLCM model may include Quantization, employing a Co-occurence matrix, symmetrizing the matrix and using normalization as described in more detail in Fig. 3. The output of the GLCM model may be the normalized GLCM for each window. The moving window may slide over the speckle image and the texture metric(s) and / or based on that, texture maps may be generated. The texture maps may illustrate the texture metric(s) in the speckle image(s) and / or pattern light image(s). The texture map(s) may be created by mapping the texture metric(s) to the speckle image(s) and / or pattern light image(s).

[0229] The complete speckle image may be recreated based on the texture metrics and / or textures maps of the moving window, e.g. Fig. 4a illustrate examples of normalized GLCM matrix images for different roughness parameters of silicon.

[0230] A GLCM matrix of a rough surface may have few numbers with large magnitude around the diagonal. A large number around diagonal may show low contrast, large numbers away from diagonal may show high contrast. The intensities of speckle grains may become weaker with increasing roughness. A rough surface may have large speckle grains that tend to increase the number of similar neighbor pixels (homogeneity increase). Another explanation of this is that there may be less gray-level transition between the neighbor pixels for rough surfaces compared to smooth ones. Therefore, fewer entries with large values to the GLCM near the diagonal are obtained. On the other hand, high variations between neighbor pixels (very smooth surface) give large entries of small values to the GLCM away from the main diagonals. As the speckle grain size decrease, the elements accumulate close to the main diagonal.

[0231] Based the normalized GLCM, one or more texture metrics may be determined. Texture measures may be weighted averages of normalized GLCM resulting in a quantitative value. The one or more texture metrics may include features such as contrast, dissimilarity, homogeneity, entropy, variance, mean, correlation or the like.

[0232] Contrast may be a measure of gray-level intensity variations between the pixels to show the image intensity contrast. Elements on GLCM matrix diagonal may show no contrast with low weights. Elements away from the diagonal may show high contrast with high weights. The weights may increase exponentially away from the diagonal. Dissimilarity240674W001

[0233] 29

[0234] weights may increase linearly from the diagonal. As the difference between the two neighbor pixels increases, the weight factor increases and consequently, the contrast feature increases.

[0235] Contrast may be determined using the following formula:

[0236]

[0237] Mean and variance may measure local variation in GLCM matrix. Mean and variance of the GLCM may not measure its frequency of occurrence by itself. Mean and variance may measure frequency of its occurrence in combination with neighbor pixel value.

[0238] Mean and variance may be determined using the following formula:

[0239]

[0240] Energy may measure the uniformity and / or orderliness of the GLCM matrix values. For determining the energy, the weight of itself may be used.

[0241] Energy may be determined using the following formula:

[0242]

[0243] Homogeneity as one of the texture metrics may be the inverse of contrast. Homogeneity may measure the closeness of the distribution of elements in the GLCM to the GLCM diagonal. Homogeneity may be determined using the following formula:

[0244]

[0245] The output of the model may be the texture metrics for each window. For each window, one texture metric may be determined. Based on the texture metric(s), the surface roughness may be extracted. For the moving window and the one or more texture metrics, one or more texture maps may be generated. For generating the one or more texture map(s), the texture metrics for the one or more window(s) may be mapped back to the image. As a result, the texture map(s) may show the input image(s) with the one or more texture metrics e.g. contrast metrics image(s). Examples of texture map(s) 504,504 are shown in Fig. 4a. Based on the texture metric(s) and the related texture map(s) one or more skin roughness measure(s) may be determined. The higher the contrast illustrated in the texture map(s), the lower the surface roughness may be. The higher the contrast illustrated in the texture map(s), the higher the skin smoothness of the surface may be. The lower the contrast illustrated in the texture map(s), the higher the surface roughness may be. The lower the contrast illustrated in the texture map(s), the lower the skin smoothness of the surface may be. The skin smoothness may be the inverse of the skin roughness.

[0246] Fig. 5 illustrates an example of a process for determining a skin smoothness score and derive a skin type based on surface roughness measures.

[0247] One or more input images may relate to one or more speckle image(s) and / or pattern light image(s). The one or more speckle image(s) and / or pattern light image(s) 501 may be captured as described in Fig. 1. The speckle240674W001

[0248] 30

[0249] image(s) and / or pattern light image(s) 501 may be fed to the analytical model(s) 502, which is described in detail in Fig. 2 and Fig. 3. Based on the analytical model(s) 502 the texture metric(s) 503 may be derived. The texture metric may relate to speckle parameter(s).

[0250] Based on the texture metric(s) 503, texture maps may be generated. For generating the one or more texture map(s) 504, 505, the texture metrics 503 for the one or more window(s) may be mapped back to the speckle image(s) and / or pattern light image(s) 501. Texture map(s) may be the mapped image(s) with speckle parameter values. Based on the texture metric(s) and the related texture map(s) one or more skin roughness measurement(s) may be determined. In the example of Fig. 4, contrast texture metrics may be determined and texture map(s) 504, 505 may be contrast texture map(s). The region indicated by a black window may be the region of interest. This region may be the input for the analytical model. The speckle pattern image may be cropped. Contrast texture map 504 may be a texture map before applying hydration. Contrast texture map 505 may be a texture map after applying hydration. When comparing the speckle value(s) of the contrast texture maps 504 and 505 increasing or decreasing speckle values may be determined. Increased speckle values such as increased contrast texture metric(s) may relate to decreased surface roughness measurement(s). Therefore, a surface roughness score of the living organism may be decreased. This may be related to an increased smoothness of the surface of the living organism such as skin. Decreased speckle values such as decreased contrast texture metric(s) may relate to increased surface roughness measurement(s). Therefore, a surface roughness score of the living organism may have increased. This may be related to a decreased smoothness of the surface of the living organism such as skin when comparing the image(s). Based on the determined surface roughness, one or more skin type(s) may be derived. Skin type(s) may relate to the SC-Layer as described in detail in Fig. 2. For determining the skin type(s) an average skin roughness measure of at least part of one or more object(s) may be determined. The average skin roughness measure before a treatment of the at least part of the one or more object(s) may be determined. A second speckle image of the same at least part of the one or more object(s) may be generated. For determining the skin type(s) an average surface roughness measure of at least part of one or more object(s) may be determined.

[0251] As a treatment hydration such as moisturizer may be applied to the skin. After the treatment a second speckle image of the same at least part of the one or more object(s) such as the region may be generated. For determining skin type(s) an average surface roughness measure of the second at least part of one or more object(s) may be determined. The difference of the average surface roughness measure may be calculated. Based on the difference, a percentage of the water content of the SC layer may be determined. The percentage of the water content may relate to the skin type of the object(s) such as dry, normal or oily. The percentage of the water content may relate to the absorption capacity of the skin. A smoothness score may be used to determine the percentage of absorption of the hydration of the skin. For example, the percentage rise for dry skin is small since the absorption of hydration of dry skin is high. The smoothness score after the application of hydration may have a slight increase e.g. 15 to 20 %. For oily skin the absorption of the hydration is small. Therefore, the percentage difference may be a slight increase or even a slight decrease of the score e.g. plus or minus 2-3%. For normal skin absorption of hydration is moderate, the percentage rise will be large, e.g. >20%. The percentage scores may vary and are mere examples. The percentage scores may vary dependent on external presence of substances on the skin prior to the application of topical240674W001

[0252] 31

[0253] hydration. For example, the percentage values may vary for any type of the skin if the makeup substances may be present on the skin prior to application of hydration.

[0254] Fig. 6 illustrates an example of a method for monitoring one or more skin characteristic® based on one or more surface roughness measure(s) associated with at least a part of one or more object(s).

[0255] In block 602 one or more speckle image(s) and / or pattern light image(s) associated with at least a part of one or more object(s) may be provided by one or more device(s) including one or more camera(s), wherein the at least part of the one or more object(s) is being illuminated by one or more illumination source(s). The one or more illumination source(s) may comprise coherent electromagnetic radiation associated with a wavelength between 850 nm and 1400 nm. The device(s) may include mobile electronic device(s). Speckle may relate to an optical phenomenon caused by interfering coherent electromagnetic radiation due to non-regular or irregular surfaces. Speckles may appear as contrast variations in an image such as a speckle image. Speckle image may refer to an image showing a plurality of speckles. The speckle image may show a plurality of speckles. The speckle image may be generated while the object may be illuminated by coherent electromagnetic radiation associated with a wavelength between 850 nm and 1400 nm. The speckle image may show a speckle pattern. The speckle pattern may specify a distribution of the speckles. The speckle image may indicate the spatial extent of the speckles. The speckle image may be suitable for determining a surface roughness measure. The speckle image may be generated with a camera. For generating the speckle image, the object may be illuminated by the illumination source. The one or more speckle image(s) and / or pattern light image(s) may be generated during a face authentication process. The one or more speckle image(s) and / or pattern light image(s) may show the same at least part of object(s). The one or more speckle image(s) and / or pattern light image(s) may be generated at a predefined time interval and / or before and / or after a treatment of the respective at least part of the object(s).

[0256] In block 604 one or more surface roughness measure(s) based on the one or more speckle image(s) and / or pattern light image(s) may be determined. Two or more speckle image(s) and / or pattern light image(s) may be provided for determining differences of the surface roughness measure(s). Surface roughness measure(s) may be determined based on a GLCM model as described in the context of Fig. 4. Based on the output of the GLCM model one or more texture metric(s) may be determined. The textures metrics(s) may indicate the surface roughness measure(s). The one or more texture metrics may include features such as contrast, dissimilarity, homogeneity, entropy, variance, mean, correlation or the like. Texture map(s) may be created by mapping the texture metric(s) to the speckle image and / or pattern light image(s). The complete speckle image(s) and / or pattern light image(s) and / or a predefined part of speckle image(s) and / or pattern light image(s) may be recreated based on the texture metrics and / or textures maps of the moving window.

[0257] In block 606 one or more skin characteristic(s) associated with a skin to be analyzed of the one or more object(s) based on one or more surface roughness measure(s) of the one or more speckle image(s) and / or pattern light image® may be determined. Skin characteristic® may include one or more parameter® associated with static skin characteristic® and / or one or more parameters associated with dynamic skin characteristic®. For determining240674W001

[0258] 32

[0259] dynamic skin characteristic® two or more speckle image(s) and / or pattern light image(s) may be provided. The two or more speckle image(s) and / or pattern light image(s) may be provided at a predefined time interval and / or before and after a treatment such as applying a moisturizer. Predefined time interval may relate to a short time interval based on the skin characteristic®. Short time interval may relate to less than 15 minutes, preferably less than 10 minutes, more preferably less than 3 minutes.

[0260] Predefined time interval may relate to a long time interval based on the skin characteristic(s). long time interval may relate to less than a year, preferably less than 30 days, more preferably less than 10 days. Skin characteristic(s) may be determined based on difference of the surface roughness measure(s).

[0261] In block 608 one or more skin characteristic(s) associated with at least a part of one or more object(s) may be provided. Based one or more skin characteristic(s) associated with at least a part of one or more object(s) a recommendation may be provided. The recommendation may include moisturizing instruction(s) and / or product recommendation® depending on skin characteristic® such as the individual's skin type.

[0262] Fig. 7 illustrates an example of a process for generating skin health data and generation authentication data.

[0263] Pattern light images 701 may be captured showing at least part of the user. Pattern light images may be captured by a system as described in the context of Figure 1. The one or more captured pattern light images 701 may be provided to at least one authentication process 702 and / or at least one skin detection process 705. The pattern light images may be provided to the authentication process(es) 702 and the skin detection process(es) 705. The pattern light images may be provided simultaneously or sequentially to the authentication process(es) 702 and the skin detection process(es) 705. The authentication process may be triggered by a user to authorize operations on the device. In addition to the pattern light images, flood images 703a may be provided to the authentication process 702. For an authentication process 702 reference data 703b may be provided. The authentication process 702 may perform at least one face detection using the flood image. Face identification, i.e. assigning an identity to the detected face may be performed using reference data 703b. The authentication unit may be configured for identifying the user based on the flood image. Particularly therefore, the authentication unit may forward data to a remote device. Alternatively, or in addition, the authentication unit may perform the identification of the user based on the flood image, particularly by running an appropriate computer program having a respective functionality. Identifying may relate to assigning an identity to a detected face and / or at least one identity check and / or verifying an identity of the user. The authentication using the flood image may be validated using the pattern light image 701 based on a surface roughness measure®. The authentication process may comprise determining if the surface roughness measure corresponds to a surface roughness measure of a human being. For example, the method may comprise determining if the surface roughness measure corresponds to a surface roughness measure of the specific user. Determining if the surface roughness measure corresponds to a surface roughness measure of a human being and / or of the specific user may comprise comparing the surface roughness measure to at least one predefined or predetermined range of values of surface roughness measure, e.g. stored in at least one database e.g. of the device or of a remote database such as of a cloud. The predefined or predetermined range of values of surface roughness measures may be part of the240674W001

[0264] 33

[0265] reference data 703b. In case the determined surface roughness measure is at least within tolerances within the redefined or predetermined range of values of surface roughness measure, the user is authenticated, otherwise the authentication is unsuccessful. The result of the authentication process may be provided as authentication data 704. Pattern light images 701 may be provided to a skin detection process 705. Based on the skin detection process 705 skin characteristics 706 associated with the skin of the user may be generated. Skin characteristics 706 may relate to one or more attribute(s) and / or feature(s) of the skin. The attribute(s) and / or feature may define its appearance, texture, color, and / or overall condition. Skin characteristics may relate to the surface roughness of the skin of the user. The surface roughness may be measured as described in Fig. 1 Fig. 5 and / or Fig. 6.

[0266] Based on the skin characteristics 706 and contextual data, skin health data 711 may be determined. Contextual data may relate to sensor data such as location data 708 and / or health data 709. In an example the surface roughness of the skin of the user indicating a high skin roughness may be determined as skin characteristics 701. In addition, the user may track his heart rate and workout data via a wearable (e.g. shown in Fig. 8) as health data 711. Location data and / or weather data indication a high UV index 708 may be provided. Based on the high skin roughness, high heart rate and the high UV index, the skin health data for the user may be determined.

[0267] The skin health data 711 may be provided for monitoring and / or tracking skin health. The skin health data may be shown to the user on a display of the mobile device. In an example, an alert may be generated stating "Low Skin health data. Get out of the sun, drink some water and / o apply sunscreen!”.

[0268] Fig. 8 illustrates an example of a device and sensors for generating skin health data.

[0269] The surface roughness measurement system 803 may be integrated into a mobile device such as a smartphone 800. In another example the surface roughness measurement system 803 may be integrated in a wearable 820 (not shown). The surface roughness measurement system 803 may be operatively controlled by an operating system 804. The surface roughness measurement system 803 may comprise an illumination source, a camera and a processor as described in Fig. 1. In an example, the processor may be physically integrated into the surface roughness measurement system 803. In another example the processor 802 may be the processor of the smartphone 800. The processor 802 may be configured to execute instructions for determining skin health by evaluating surface roughness measurements in combination with contextual data 805. The smartphone may comprise a temperature sensor 808 and / or an altitude sensor 809. The sensors may detect data, which may be made available to processor 802 of the skin health determination 801 as contextual data 806 via the operating system 804.

[0270] A remote photoplethysmography (rppg) sensor 821 for heart rate detection may be disposed on the wearable device 820. The rppg data may be stored on the wearable 820 and may be transfered to a an rppg data interface 801 via a bluetooth conntection to the smartphone and may be made available to the processor 802 of the skin health determination 801 as contextual data 806 via the operating system 804. Contextual data 806 may be continuously or intermittently provided by one or more sensors and made available to processor 802 for skin health determination 801 through the operating system 804.240674W001

[0271] 34

[0272] Location data 807 may be acquired from sources including, but not limited to, global positioning system (GPS), Wi-Fi, or Bluetooth technologies, wherein Wi-Fi-based location acquisition may utilize data regarding proximate wireless networks. The location data 807 may be made accessible via operating system 804 and may be utilized to retrieve weather data. In certain embodiments, a data service provider 840, such as a weather channel, may acquire environmental data 841, including weather data, based on the location data. The weather data, determined according to the location data 807, may be communicated to the smartphone via a Wi-Fi connection and subsequently made available to processor 802 for skin health determination 801 as contextual data 806 via the operating system 804.

[0273] The present disclosure has been described in conjunction with preferred embodiments and examples as well. However, other variations can be understood and effected by those persons skilled in the art and practicing the claimed invention, from the studies of the drawings, this disclosure and the claims.

[0274] Any steps presented herein can be performed in any order. The methods disclosed herein are not limited to a specific order of these steps. It is also not required that the different steps are performed at a certain place or in a certain computing node of a distributed system, i.e. each of the steps may be performed at different computing nodes using different equipment / data processing.

[0275] As used herein ..determining" also includes ..initiating or causing to determine", "generating" also includes ..initiating and / or causing to generate" and "providing” also includes "initiating or causing to determine, generate, select, send and / or receive”. "Initiating or causing to perform an action” includes any processing signal that triggers a computing node or device to perform the respective action.

[0276] In the claims as well as in the description the word "comprising” or "including” or similar wording does not exclude other elements or steps and shall not be construed limiting to the elements or steps lined out. The indefinite article "a” or "an” does not exclude a plurality. A single element or other unit may fulfill the functions of several entities or items recited in the claims. The mere fact that certain measures are recited in the mutual different dependent claims does not indicate that a combination of these measures cannot be used in an advantageous implementation or further elements may be included.

[0277] Providing in the scope of this disclosure may include any interface configured to provide data. This may include an application programming interface, a human-machine interface such as a display and / or a software module interface. Providing may include communication of data or submission of data to the interface, in particular display to a user or use of the data by the receiving node, entity or interface.

[0278] Various units, circuits, entities, nodes or other computing components may be described as "configured to” perform a task or tasks. Configured to shall recite structure meaning "having circuitry that” performs the task or tasks on operation. The units, circuits, entities, nodes or other computing components can be configured to perform the task even when the unit / circuit / component is not operating. The units, circuits, entities, nodes or other computing components that form the structure corresponding to "configured to” may include hardware circuits and / or memory storing program instructions executable to implement the operation. The units, circuits, entities, nodes or other computing240674W001

[0279] 35

[0280] components may be described as performing a task or tasks, for convenience in the description. Such descriptions shall be interpreted as including the phrase "configured to.” Any recitation of "configured to” is expressly intended not to invoke 35 U.S.C. § 112(f) interpretation.

[0281] In general, the methods, apparatuses, systems, computer elements, nodes or other computing components described herein may include memory, software components and hardware components. The memory can include volatile memory such as static or dynamic random-access memory and / or nonvolatile memory such as optical or magnetic disk storage, flash memory, programmable read-only memories, etc. The hardware components may include any combination of combinatorial logic circuitry, clocked storage devices such as flops, registers, latches, etc., finite state machines, memory such as static random-access memory or embedded dynamic random-access memory, custom designed circuitry, programmable logic arrays, etc.

[0282] Any disclosure and embodiments described herein relate to methods, systems, apparatuses, devices, materials, computer program elements lined out above and vice versa. Advantageously, the benefits provided by any of the em-bodiments and examples equally apply to all other embodiments and examples and vice versa.

[0283] All terms and definitions used herein are understood broadly and have their general meaning.

Claims

240674W00136CLAIMS1. A method for monitoring and / or tracking skin health based on one or more pattern light image(s):triggering an authentication process at least by illuminating at least part of an user provided by at least one infrared pattern illuminator;capturing image(s) including one or more pattern light image(s) of at least part of the user;providing the captured image(s) toat least one authentication process configured to authenticate the user by generating authentication data based on captured image(s) andat least one skin detection process configured to determine skin health data by generating one or more skin characteristic(s) associated with a skin of the user based on the captured one or more pattern light image(s);providing at least the skin health data for monitoring and / or tracking skin health.

2. The method of any of the preceding claims, wherein the authentication process is triggered by at least one unlock event related to the user.

3. The method of any of the preceding claims, wherein the skin detection process includes determining one or more surface roughness measure(s) based on the captured one or more pattern light image(s) and to generate the one or more skin characteristic(s) based on the one or more surface roughness measure(s).

4. The method of any of the preceding claims, wherein the skin detection process includes determining the level of roughness associated with the skin or an area of skin of the user by analyzing the captured one or more pattern light image(s) with respect to textural features of the skin's outermost epidermal layer and generating one or more skin characteristic(s) related to a smoothness of the skin or an area of skin of the user.

5. The method of any of the preceding claims, wherein the skin detection process includes determining skin health data by detecting at least one region of interest in the captured one or more pattern light image(s) and generating one or more skin characteristic(s) for the at least one region of interest associated with at least one area of the user's skin.

6. The method of any of the preceding claims, wherein the skin detection process includes determining skin health data based on one or more contextual data.

7. The method of any of the preceding claims, wherein contextual data comprises location data associated with the location of the user and / or health data associated with the user.240674W001378. The method of any of the preceding claims, wherein the skin detection process includes determining skin health data based on historically generated skin characteristic(s) determined from one or more pattern light image(s) captured during previous authentication processes, wherein the skin detection process includes determining skin health data based on location data associated with the current location of the user.

9. The method of any of the preceding claims, wherein skin health data is associated with at least one time dependent change in skin health and / or at least one treatment instruction.

10. The method of any of the preceding claims, wherein the capturing of one or more pattern light image(s) to be provided to the at least one authentication process and the at least one skin detection process is executed in the same step.

11. The method of any of the preceding claims, wherein skin health data relates to time-dependent changes in skin health, wherein a notification to the user is provided, if a change in skin health is detected by the skin detection process.

12. The method of any of the preceding claims, wherein skin health data relates to time-dependent change in skin health and / or at least one treatment instruction, wherein the at least one treatment instruction is generated based on the time-dependent change in skin health, wherein the at least one treatment instruction is provided, if a change in skin health is detected by the skin detection process.

13. The method of any of the preceding claims, wherein skin health data relates to time-dependent change in skin health and / or at least one treatment instruction, wherein at least one treatment instruction is generated based on the contextual data.

14. A mobile electronic device comprising at least one infrared pattern illuminator configured to illuminate at least part of an user, one or more camera(s) configured to capture at least a part of an user illuminated by the at least one infrared pattern illuminator, one or more processor(s) configured to monitor and / or track skin health based on one or more pattern light image(s) and one or more display(s) configured to display skin health data, the device being configured to:trigger, by at least one processor, an authentication process at least by illuminating at least part of an user provided by at least one infrared pattern illuminator;capture, by at least one camera, at least one or more pattern light image(s) of at least part of the user; providing the captured one or more pattern light image(s) to at least one processor configured to execute at least one authentication process configured to authenticate the user by generating authentication data based on at least the captured one or more pattern light image(s) and240674W00138at least one skin detection process configured to determine skin health data by generating one or more skin characteristic(s) associated with a skin of the user based on the captured one or more pattern light image(s);providing at least the skin health data for monitoring and / or tracking skin health to the at least one display.

15. Use of skin health data determined according to the methods of any one of claims 1 to 14 for displaying one or more treatment recommendation(s) to a user of an electronic mobile device.

Images