Automated Skin Symptom Assessment

Abstract

Aspects of the present disclosure relate to automated skin condition assessment. A region of interest within a user's image data that is affected by a skin or nail condition is identified. A control region not affected by the condition may also be identified, and a set of chromophore concentrations is generated for the control region. The set of chromophore concentrations is used to generate an estimated thickness for the region of interest. Thus, it is assumed that the amount of chromophore in the control region is at least approximately the same as the amount of chromophore in the region of interest, such that any differences in color between the control region and the region of interest are primarily due to differences in thickness. Thus, ultimately, the estimated thickness and / or surface roughness may be used to generate a skin / nail condition severity metric for the region of interest.

Description

[Background technology]

[0001] Certain conditions affecting human skin and / or nails can increase the thickness of the skin and / or nails. For example, psoriasis of the skin or nails can cause hyperkeratosis (excessive thickness of skin tissue) as well as pitting or other visible signs of the skin or nails. However, objectively assessing the severity and / or progression of such conditions can be difficult. Similarly, while treatments have been developed and can be prescribed to address these and other conditions, objectively assessing the effectiveness of the treatment on the affected areas can be difficult.

[0002] It is with respect to these and other general considerations that the embodiments are described, and it should be understood that although relatively specific problems have been discussed, the embodiments should not be limited to solving the specific problems identified in the background of the invention. Summary of the Invention

[0003] Aspects of the present disclosure relate to automated skin condition assessment. In an embodiment, a region of interest affected by a skin or nail condition is identified within a user's image data. A control region not affected by the condition may also be identified, and a set of chromophore concentrations is generated for the control region. The set of chromophore concentrations is used to generate an estimated thickness for the region of interest. Thus, it is assumed that the amount of chromophore in the control region is at least approximately the same as the amount of chromophore in the region of interest, so that any differences in color between the control region and the region of interest are primarily due to differences in thickness (since chromophore concentration is expected to decrease as skin thickness increases if the amount of chromophore is held constant).

[0004] Furthermore, because some skin conditions may alter the roughness or texture of the skin without significantly affecting the skin's associated thickness, surface roughness may additionally or alternatively be determined for the region of interest. The estimated thickness and / or surface roughness may be used to generate a skin / nail condition severity metric for the region of interest. An indication of such thickness, surface roughness, and / or condition severity metric may ultimately be provided to the user, thereby enabling an objective determination of whether a skin / nail condition is present. Additionally or alternatively, this may enable objective tracking of the progression of a skin / nail condition over time.

[0005] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to determine the scope of the claimed subject matter. [Brief explanation of the drawings]

[0006] Non-limiting and non-exhaustive examples are described with reference to the following figures: [Figure 1] 1 illustrates an overview of one exemplary system in which automated skin condition assessment may be performed according to aspects described herein. [Figure 2A] 1 illustrates exemplary image data of a user's hand that may be evaluated according to the disclosed automated skin condition assessment techniques. [Figure 2B] 1 illustrates an exemplary control area used for automated skin condition assessment according to embodiments described herein. [Figure 2C] 2B illustrates an exemplary region of interest that may be evaluated based on the control region of FIG. 2B according to embodiments described herein. [Figure 3A] 1 illustrates an overview of one example method for generating a thickness estimate for a region of interest according to aspects described herein. [Figure 3B] 10 illustrates an overview of another exemplary method for generating a thickness estimate for a region of interest according to aspects described herein. [Figure 4A] 1 illustrates an overview of an exemplary method for determining symptom severity based on estimated thickness and estimated roughness, according to aspects described herein. [Figure 4B] 1 illustrates an overview of one exemplary method for determining whether a skin condition is present, according to aspects described herein. [Figure 5] 1 illustrates an example of a suitable operating environment in which one or more aspects of the present application may be implemented. DETAILED DESCRIPTION OF THE INVENTION

[0007] In the following detailed description, references are made to the accompanying drawings that form a part hereof, and in which specific embodiments or examples are shown, by way of illustration. These aspects may be combined, other aspects may be utilized, and structural changes may be made without departing from the disclosure. The embodiments may be embodied as methods, systems, or devices. Thus, the embodiments may take the form of a hardware implementation, an entirely software implementation, or an implementation combining software and hardware aspects. Therefore, the following detailed description is not intended to be taken in a limiting sense, and the scope of the present disclosure is defined by the appended claims and their equivalents.

[0008] Certain conditions affecting a person's skin and / or nails can increase the thickness of the skin and / or nails. As another example, such conditions can cause pitting or other visible signs in the skin or nails. While treatments to address these and other conditions have been developed and can be prescribed, objectively assessing the severity / progression of such conditions and the effectiveness of treatments can be difficult. For example, an individual's skin and / or nails can be manually assessed (e.g., by the individual or by a supervisor), but such assessments can produce inconsistent results between individuals, and in some cases, even between observations on the same individual, due to the subjectivity of manual assessment.

[0009] Accordingly, aspects of the present disclosure relate to automated skin condition assessment. In embodiments, image data of a user is captured (e.g., including any of the user's hands, feet, nails, arms, legs, or various other skin and / or nail regions). Within the image data, a region of interest is identified (e.g., manually or automatically), which region of interest is affected by a skin or nail condition. In embodiments, a region not affected by the skin or nail condition is similarly identified within the image data (referred to herein as a "control region").

[0010] A set of chromophore concentrations is generated for the control region based on the image data for the control region. In embodiments, the model that generates the set of chromophore concentrations based on the image data for the control region uses a predetermined or otherwise assumed tissue (e.g., skin / nail) thickness for the control region. An estimated tissue thickness is then generated for the region of interest based on the set of chromophore concentrations generated for the control region.

[0011] This estimate of tissue thickness of the control region can be generated by applying the above-described model in reverse. Thus, it is assumed that the chromophore concentration of the control region is applicable to the region of interest, such that any differences in color between the control region and the region of interest are primarily due to differences in thickness (since, assuming substantially similar amounts of chromophore, the chromophore concentration in the region of interest is expected to decrease as skin thickness increases). It will be appreciated that the resulting estimated thickness of the region of interest can therefore be at least a relative measurement (e.g., relative to the control region) that can be used to identify, among other examples, thickness differences between the control region and the region of interest, as well as changes in thickness over time. Similarly, an indication of the estimated thickness of the region of interest can be provided as a relative measurement compared to the control region.

[0012] Surface roughness may be additionally or alternatively measured for a region of interest. For example, some skin conditions (e.g., pruritus nodularis) may alter the roughness or texture of the skin without significantly affecting the associated thickness of the skin. Other conditions may alter both tissue thickness and roughness. Thus, assessing surface roughness in addition to or as a substitute for thickness may enable the assessment of a wider range of skin / nail conditions.

[0013] The thickness and / or surface roughness determined according to aspects described herein can be used to generate a skin / nail symptom severity metric. For example, the symptom severity metric can be generated based on a combination of the thickness and surface roughness of the region of interest. This combination can be generated based on a thickness weighting and a roughness weighting, such that the symptom severity metric is a weighted average of the generated thickness and roughness metrics. As another example, the symptom severity metric can be generated using a machine learning model (e.g., using annotated training data and / or reinforcement learning) trained to generate the symptom severity metric for a set of inputs, including the determined thickness and / or surface roughness of the region of interest. As a further example, the symptom severity metric can be generated based on multiple thickness and / or surface roughness determinations (e.g., for multiple regions of interest and / or for multiple measurements on the same region of interest).

[0014] The generated thickness, surface roughness, and / or symptom severity metrics may be used in any of a variety of subsequent processes, according to aspects described herein. For example, an indication of one or more of such metrics may be provided to a user, allowing the user to determine whether a skin / nail condition is present and / or track the progression of that condition over time. As another example, such metrics may be generated for a population of users based on image data collected for each user over time, such that the effectiveness of skin / nail treatments may be objectively quantified and assessed across a population of users.

[0015] It will be appreciated, therefore, that aspects of the present disclosure may be implemented using recently captured image data (e.g., contemporaneous with the disclosed processing) and / or retroactively (e.g., based on previously captured image data), among other examples. As another example, a first set of thickness, surface roughness, and / or symptom severity metrics may be generated (e.g., contemporaneous with the time the image data is captured), and a second set of metrics may be generated for this image data at a later time, such as when a different or improved model becomes available for processing the image data and generating one or more associated metrics.

[0016] A set of chromophore concentrations can be generated for the control region and / or region of interest according to any of a variety of techniques. It will be understood that any of a variety of wavelength bands can be used. For example, using spectral reconstruction or spectral super-resolution, any number of wavelength bands can be reconstructed (e.g., including three spectral bands, from 400 nm to 700 nm at 10 nm intervals from RGB image data). Further examples of such aspects are described in the following references, which are incorporated herein by reference in their entirety: Kaya, B., Can, YB, & Timofte, R. (2018). Towards Spectral Estimation from a Single RGB Image in the Wild. https: / / doi.org / 10.48550 / arxiv.1812.00805.

[0017] The intensity values ​​in each wavelength band (e.g., in the red, green, and blue channels) may correspond to the respective concentrations of various chromophores (e.g., melanin, oxygenated hemoglobin, deoxygenated hemoglobin, and bilirubin), whereby the processed wavelength bands are further processed (e.g., based on the absorption coefficient of each chromophore, which may be related to tissue thickness) to generate a set of chromophore concentrations accordingly.

[0018] For example, a set of wavelength band intensity values ​​(e.g., for red, green, and blue channels, and / or for bands determined using spectral reconstruction or spectral super-resolution) may be multiplied by a color transformation matrix N1, thereby converting the set of wavelength band intensity values ​​to another color system or other set of wavelengths. The other color system or other set of wavelengths may have any number of dimensions m. As an example, N1 may be mathematically determined based on one or more known techniques for converting from one color system to another. The resulting set of wavelength band intensity values ​​is referred to herein as a spectral vector.

[0019] In some embodiments, the resulting spectral vector is multiplied by an estimation matrix N2, thereby generating a vector of chromophore concentrations (also referred to herein as a set of chromophore concentrations) corresponding to the chromophore space. In one embodiment, the estimation matrix is ​​determined using a computer model of human skin (e.g., including the stratum corneum, epidermis, and dermis). In one such embodiment, the computer model may be programmed with absorption coefficients corresponding to various chromophores, which may be obtained from the literature. Alternatively or additionally, N2 may be determined (and / or validated) using experimental data, among other embodiments.

[0020] Further examples of such aspects are described in the following documents, which are incorporated herein by reference in their entirety: Nishidate, I., Minakawa, M., McDuff, D., Wares, MA, Nakano, K., Haneishi, H., Aizu, Y., & Niizeki, K. (2020). Simple and affordable imaging of multiple physiological parameters with RGB camera-based diffuse reflectance spectroscopy. Biomedical Optics Express, 11(2), 1073-1091. https: / / doi.org / 10.1364 / BOE.382270.

[0021] In another example, a method is provided for determining whether skin in a region of interest may exhibit abnormal thickness (e.g., compared to a control region). In an example, a vector C (e.g., in an RGB color system) is calculated for the control region.

number

number

number

[0022] In an embodiment, f(C) and g(X) may each be approximately invertible. For example, if N1 is a square nonsingular matrix, then the inverse function of f(C) is therefore f -1 (X)=N1 -1 X, and thus we can solve the inverse problem N1C=X for the unknown C. Similarly, if N2 is a square nonsingular matrix, the inverse function of g(X) is therefore g-1 (Y)=N2 -1 Y, and thus we can solve the inverse problem N2X=Y for the unknown X.

[0023] Therefore, the chromophore vector of the control region can be defined according to the above function as g(f(C)) = Y, and the wavelength band intensity value C2 (C2 = f -1 (g -1 (Y))) can be used to calculate an expected set of wavelength band intensity values ​​C2 that represent the wavelengths that are expected to be observed from the region of interest if the skin tissue thickness of the region of interest is the same as that of a comparison region. Accordingly, the sets of wavelength band intensity values ​​R for the region of interest can thus be compared according to the following example formula:

[0024]

number

number

number

number

number

number

[0025]

number

[0026] In some embodiments, the difference in color between the region of interest and the control region is assumed to be due to the skin in the region of interest being thicker than the skin in the control region, and thus the thickness t calculated above is an estimate of how thick the skin in the region of interest is compared to the skin in the control region.

[0027] As another example, if functions f and g are assumed to have no inverses, then Y and Z can be calculated and evaluated instead (where Y = g(f(C)) and Z = g(f(R))). Thus, similar to the example functions above, the distance metric

number

[0028]

number

[0029] If the above equation is correct, the estimated thickness (t) of the tissue in the region of interest compared to the control region can be quantified according to the following exemplary equation:

[0030]

number

[0031] Finally, if the chromophore-space-based evaluation according to function g is omitted and function f is similarly assumed to have no inverse, then a similar comparison in spectral space can be performed based on f(C) and f(R), as above, for adjustment values ​​λ≧0.

[0032]

number

[0033] Thus, similar to the previous two example equations, if the above equation evaluates to true, the region of interest may be determined to exhibit abnormal skin thickness. If so, the estimated thickness (t) of the tissue in the region of interest compared to the control region may be quantified according to the following example equation:

[0034]

number

[0035] In some embodiments, any of the above estimates of thickness t may be further converted by some function h(t) that converts the detected difference in color development between the region of interest and the control region into units of length, such as millimeters or micrometers. The function h(t) may be determined experimentally and, in some examples, may include multiplying the thickness by an experimentally determined scaling constant.

[0036] Similarly, surface roughness may be determined for the control region and / or region of interest according to any of a variety of techniques. For example, the image data may be processed to determine a set of excerpts (e.g., cropped segments of the image having a predetermined width and / or according to a predetermined interval, which may or may not overlap). In an embodiment, the image data may be converted to grayscale to reduce the effect of color variability compared to lightness / darkness variations. Each of the determined excerpts may then be processed to evaluate the variation in pixel values ​​along the excerpt. Thus, excerpts exhibiting high variability may be determined to have a greater surface roughness than excerpts exhibiting low variability, and a surface roughness metric may be generated accordingly. Further examples of such aspects are described in the following document, which is incorporated herein by reference in its entirety: Jafari, A., Fazayeli, A., & Zarezadeh, M. (2014). Estimation of orange skin thickness based on visual texture coarseness. Biosystems Engineering, 117, 73-82. https: / / doi.org / 10.1016 / j.biosystemseng.2013.08.010.

[0037] Although various techniques for wavelength band processing, tissue thickness estimation, and surface roughness estimation have been discussed above, it will be understood that such aspects are provided as examples and that in other embodiments, any of a variety of additional or alternative techniques may be used.

[0038] 1 illustrates an overview of an example system 100 in which automated skin condition assessment may be performed according to embodiments described herein. As illustrated, system 100 includes a data processing platform 102, a computing device 104, and a network 106. In an embodiment, data processing platform 102 and computing device 104 communicate via network 106. For example, network 106 may include a local area network, a wireless network, or the Internet, or a combination thereof, among other embodiments.

[0039] As illustrated, the data processing platform 102 includes an image preprocessor 108 , a region classifier 110 , a chromophore processor 112 , a thickness evaluator 114 , a roughness determiner 115 , and a symptom determination engine 116 .

[0040] In some embodiments, the image preprocessor 108 obtains image data from the computing device 104 that has been preprocessed in accordance with aspects described herein. It will be appreciated that the image preprocessor 108 may obtain image data from any of a variety of additional or alternative sources, which may be the case when previously stored image data is processed. In some embodiments, the image preprocessor 108 generates image data corresponding to one or more wavelength bands (e.g., corresponding to red, green, and blue channels or various wavelength ranges). As another example, the image preprocessor 108 generates a grayscale representation of the image data that may be used by the roughness determiner 115. In other embodiments, the image preprocessor 108 adjusts the brightness and / or contrast of the image data to account for the lighting of the environment in which the image data was acquired (e.g., by the image capture device 118). As a further example, the image preprocessor 108 generates a spectral vector of a given set of wavelength band intensity values, e.g., based on a color transformation matrix N1, in accordance with aspects described herein. It will thus be appreciated that any of a variety of preprocessing operations may be performed by the image preprocessor 108.

[0041] Region identifier 110 processes image data to identify regions of interest and / or regions of control. In embodiments, region identifier 110 identifies regions based on user instructions received from computing device 104. For example, image processing application 120 may enable a user to identify one or more regions of interest and / or regions of control.

[0042] As another example, such regions may be identified automatically. For example, the region identifier 110 automatically identifies a control region, including skin and / or nails that are “normal” (e.g., not affected by the condition for which the treatment is being performed). In some embodiments, the control region is determined (e.g., may be determined automatically or manually) based on the region of interest, which may be the case if the control region is identified in a location on the user's body that is similar to the region of interest. As a further example, a region of relatively smooth skin or nails may be identified as the control region.

[0043] Accordingly, it will be appreciated that any of a variety of techniques may be used to identify control regions and / or regions of interest in accordance with aspects described herein. Furthermore, while such aspects have been described with respect to the region identifier 110 of the data processing platform 102, it will be appreciated that similar aspects may alternatively or additionally be implemented by the image processing application 120. For example, the image processing application 120 may perform such aspects of extracting identified regions, thereby reducing the amount of data transmitted from the computing device 104 to the data processing platform 102. It will also be appreciated that the order in which the image preprocessor 108 and the region identifier 110 perform operations on image data received from the computing device 104 may be interchangeable, according to various embodiments. In some examples, the region identifier 110 may first process image data received from the computing device 104, and then the image preprocessor 108 may process image data output from the region identifier 110. Alternatively, the image preprocessor 108 may first process the image data received from the computing device 104 , and then the region identifier 110 may process the image data output from the image preprocessor 108 .

[0044] The data processing platform 102 is further illustrated as including a chromophore processor 112 that processes image data of the control regions (e.g., regions identified by the region identifier 110 and / or preprocessed by the image preprocessor 108) in accordance with aspects described herein to generate a set of chromophore concentrations. As described above, any of a variety of techniques may be used to generate the set of chromophore concentrations. For example, mathematical transformations N1 (e.g., a color transformation matrix (to the extent not already applied by the image preprocessor 108) and N2 (e.g., an estimation matrix) may be used to evaluate the functions f(C) and g(X) described above, thereby generating a set of chromophores based on a set of wavelength bands of given image data (e.g., corresponding to the control regions and regions of interest). Exemplary chromophores include, but are not limited to, melanin, oxygenated hemoglobin, deoxygenated hemoglobin, and bilirubin. It will be understood that in other embodiments, fewer, additional, or alternative chromophores may be used.

[0045] The thickness evaluator 114 processes the set of chromophore concentrations to generate an estimated thickness corresponding to a region of interest (e.g., which may be identified by the region identifier 110). For example, the set of chromophore concentrations may correspond to a control region and may be generated by the chromophore processor 112. As another example, one or more evaluations may be performed according to the exemplary equations described above. For example, the thickness evaluator 114 may evaluate the distance between the set of wavelength band intensity values ​​for the control region C and further estimate the region of interest R by adjusting the adjustment value λ (e.g.,

number

number

number

[0043] Processing is performed according to

[0043] to determine whether the region of interest exhibits abnormal skin thickness. As a further example, a set of chromophores may be determined for a given region of interest, which may be the case when, among other examples, the data store includes known or expected chromophore concentrations for a given demographic and / or skin / nail region. It will be appreciated, therefore, that in some examples, the use of a control region is not required. In such cases, a set of wavelength band intensity values ​​C0 derived from expected chromophore concentrations for a given demographic and / or skin / nail region (e.g., as empirically measured or observed from a population of other users) may be substituted into the calculations and formulas described above in place of the set of wavelength band intensity values ​​C from the control region.

[0046] In some embodiments, as previously described, thickness estimator 114 processes pixels of image data corresponding to a region of interest based on a set of chromophore concentrations to generate a corresponding thickness estimate t or h(t). In some embodiments, multiple pixels within the region of interest are processed to generate a thickness estimate for each pixel, thereby forming a thickness map that indicates the tissue thickness gradient within the region of interest. In other embodiments, a single thickness estimate (e.g., relative to a control region) may be generated for multiple pixels within the region of interest, or even for the entire region of interest.

[0047] The roughness determiner 115 processes the region of interest to generate a surface roughness metric associated therewith. As mentioned above, any of a variety of techniques may be used to generate the surface roughness metric. For example, a grayscale representation of the region of interest (which may be generated, for example, by the image preprocessor 108) may be processed to extract one or more strips or excerpts therefrom, whereby the variation in lightness / darkness within the strips or excerpts may be used to generate a surface roughness metric accordingly (e.g., where high variability indicates increased surface roughness).

[0048] The data processing platform 102 further includes a symptom determination engine 116 that processes the estimated thickness (or thickness map) generated by the thickness evaluator 114 and / or the surface roughness generated by the roughness determiner 115 to generate a symptom severity metric. As described above, the severity metric may include a weighted average of the estimated thickness and surface roughness. In other examples, a machine learning model is used to generate a classification accordingly based on the estimated thickness and / or surface roughness. Thus, the symptom determination engine 116 evaluates the thickness map and / or surface roughness corresponding to the region of interest to determine the symptom and severity associated therewith.

[0049] The data processing platform 102 may provide an indication of the determined thickness, surface roughness, and / or condition severity metrics to the computing device 104. In other examples, the generated metrics and / or associated image data are stored in a data store for subsequent processing. For example, historical metrics may be retained for one or more individuals, thereby enabling longitudinal analysis of an individual's skin / nail condition progression. It will therefore be appreciated that the data processing techniques described herein may be used in any of a variety of contexts.

[0050] As illustrated, computing device 104 includes image capture device 118 and image processing application 120. In an embodiment, computing device 104 is a mobile computing device, a tablet computing device, or a laptop computing device. Image capture device 118 can be used to capture image data, including, but not limited to, an image and / or one or more frames of video.

[0051] As one example, image processing application 120 obtains image data from image capture device 118, which is processed (e.g., by data processing platform 102) according to aspects described herein. As one example, a user operates image processing application 120 to capture at least a portion of the individual's body using image capture device 118. In an example, the individual is the user (e.g., this may be the case when the user is using image processing application 120 for self-diagnosis). As another example, another individual may operate computing device 104 (e.g., this may be the case in a clinical trial).

[0052] The captured image data is provided to the data processing platform 102, which may receive one or more generated metrics (e.g., generated by the thickness evaluator 114, the roughness determiner 115, and / or the condition determination engine 116) in response. In an embodiment, the data processing platform 102 provides an indication of the identified skin condition and associated severity metric, which may be presented to the user of the computing device 104 by the image processing application 120 in response.

[0053] While computing device 104 is illustrated as including image capture device 118, it will be understood that in other embodiments, image capture device 118 may be another device, such as a camera, that communicates with computing device 104 and / or data processing platform 102 (e.g., via a wired connection, a wireless connection, and / or network 106). Furthermore, while system 100 is illustrated as including one data processing platform 102 and one computing device 104, it will be understood that in other embodiments, any number of such elements may be used. For example, image data from multiple devices may be processed on a single data processing platform, or, as another example, various computing devices may each have an associated data processing platform.

[0054] Furthermore, it will be understood that in other embodiments, the functionality described herein may be distributed or otherwise implemented according to any of a variety of other configurations. For example, computing device 104 may implement aspects related to data processing platform 102, whereby image data acquired by image capture device 118 is processed by image processing application 120 in addition to, or as an alternative to, processing performed by data processing platform 102. Performing data processing locally on computing device 104 may, among other embodiments, improve user privacy and may also reduce the amount of data transferred to data processing platform 102.

[0055] 2A illustrates example image data 200 of a user's hand that may be evaluated in accordance with the disclosed automated skin condition assessment techniques. For example, image data 200 may be captured by image capture device 118 as a result of a user operating image processing application 120 on computing device 104 of FIG. 1. While image data 200 shows an individual's hand, it will be understood that in other embodiments, any of a variety of other areas of a user's body may be evaluated.

[0056] 2B and 2C , image data 200 may be processed according to aspects described herein to identify control region 202 and region of interest 204. For example, regions 202 and 204 may be automatically identified by a region identifier, such as region identifier 110 discussed above with respect to FIG. 1 . As another example, user input may be received (e.g., by an image processing application, such as image processing application 120) that includes an indication of a boundary corresponding to control region 202 and / or a boundary corresponding to region of interest 204. As a further example, user input may indicate a general region of image data 200, which is processed by the region identifier to identify more specific control regions and / or regions of interest therein.

[0057] As illustrated, a nail in control region 202 in Figure 2B is processed according to embodiments described herein to generate a set of chromophore concentrations, which is used to generate a nail thickness in region of interest 204 in Figure 2C. In some embodiments, a thickness map may be generated that indicates various (e.g., increased) thicknesses corresponding to region 206 illustrated in Figure 2C. Thus, it may be determined whether the nail abnormality in illustrated region 206 is due to differences in nail thickness or some other condition.

[0058] 3A illustrates an overview of an example method 300 for generating a thickness estimate for a region of interest according to aspects described herein. In an embodiment, aspects of method 300 are performed by a data processing platform, such as data processing platform 102 of FIG. 1 .

[0059] Method 300 begins at operation 302, in which RGB image data of a dermatological region is obtained. For example, the image data is obtained from an image capture device of a computing device, such as image capture device 118 of computing device 104 of FIG. 1. As another example, the image data is obtained from a data store (e.g., of a data processing platform). Thus, it will be appreciated that the image data may be obtained from any of a variety of sources.

[0060] At operation 304, the acquired image data is decomposed into discrete wavelength bands (e.g., illustratively 400 nm to 700 nm). Aspects of operation 304 may be performed by an image preprocessor, such as image preprocessor 108 of FIG. 1. In embodiments, the discrete wavelength bands include corresponding spectral vectors in spectral space, which may be generated based on a color transformation matrix (e.g., according to function f(X) and color transformation matrix N1, described above). Next, at operation 306, one or more regions of interest and control regions are extracted. As described above, the extracted regions may be identified automatically and / or based on user input (e.g., by a region classifier, such as region classifier 110 of FIG. 1).

[0061] Method 300 proceeds to operation 308, where image data corresponding to the extracted control region(s) is processed to generate a set of chromophore concentrations. As illustrated, in some embodiments, a model including a set of coefficients corresponding to a predetermined thickness may be used to generate the set of chromophore concentrations. In some embodiments, such an embodiment may be performed by a chromophore processor (e.g., chromophore processor 112 of FIG. 1). For example, an estimation matrix N2 is used to transform the spectral vector, thereby generating the set of chromophores (e.g., according to the function g(X) described above).

[0062] As a result, one or more pixels within the region(s) of interest extracted in act 306 are processed in act 310 to generate a corresponding estimated thickness (e.g., an estimated thickness t or h(t), which may be generated by a thickness estimator, such as thickness estimator 114 of FIG. 1 . As described above, the set of chromophore concentrations generated in act 308 may be used to determine the estimated thickness, whereby the amount of chromophore is assumed to be substantially similar while allowing for thickness variation, since as skin thickness increases, chromophore concentration may decrease (thereby solving the inverse problem processed in act 308). As another example, in act 308, one or more comparisons (e.g., based on a color transformation matrix and / or an estimation matrix) are performed to determine whether the region of interest exhibits an abnormal skin thickness, as discussed above with respect to thickness estimator 114 and associated example equations.

[0063] In operation 312, a thickness map is generated based on the thickness estimates (e.g., t or h(t)) generated in operation 310. For example, multiple pixels or other subparts of the region of interest can be processed to generate corresponding thickness estimates, such that the thickness map generated in operation 312 includes each thickness estimate.

[0064] Flow continues to operation 314, where gradient information is extracted from the thickness map and, accordingly, this information is used to determine a symptom of the region of interest. In embodiments, aspects of operation 314 are performed by a symptom determination engine, such as symptom determination engine 116 of FIG. 1 . Accordingly, operation 314 may determine associated symptom and / or symptom severity metrics, among other embodiments. In some cases, operation 314 includes providing (e.g., to a computing device, such as computing device 104) an indication of the determined symptom and / or severity. As another example, operation 314 includes storing the generated metric and / or associated data in a data store for subsequent processing. Method 300 ends at operation 314.

[0065] 3B illustrates an overview of another example method 350 for generating a thickness estimate for a region of interest according to aspects described herein. In an embodiment, aspects of method 350 are performed by a data processing platform, such as data processing platform 102 of FIG. 1 .

[0066] Method 350 begins with operation 352, in which image data is acquired. For example, the image data may be acquired from a data store or from an image capture device (e.g., image capture device 118 of computing device 104 of FIG. 1), among other examples. The image data includes at least a portion of a user's body, such as one or more areas affected by a skin / nail condition. Aspects of operation 352 may be similar to those discussed above with respect to operation 302 of method 300 of FIG. 3A, and therefore need not be described in detail again.

[0067] Flow continues to operation 354, where the image data acquired in operation 352 is preprocessed. For example, the image data may be decomposed into its component red, green, and blue channels. As another example, image data corresponding to one or more wavelength bands is generated. Aspects of operation 354 may be performed by an image preprocessor, such as image preprocessor 108 of FIG. 1. Aspects of operation 354 may be similar to those previously discussed with respect to operation 304 of method 300 of FIG. 3A, and therefore need not be described again in detail.

[0068] Operation 354 is illustrated with a dashed box to indicate that in some embodiments, operation 354 may be omitted. For example, the image data acquired in operation 352 may not require pre-processing, which may be the case if the image data is captured under controlled or consistent conditions or if image data corresponding to one or more wavelength bands is already provided, among other embodiments.

[0069] At operation 356, a control region and a region of interest are determined. Aspects of operation 356 may be similar to those previously discussed with respect to operation 306 of method 300 of FIG. 3A and therefore need not be described in detail again. For example, the regions may be automatically identified or determined based on received user instructions (e.g., corresponding to user input in an application, such as image processing application 120), among other examples. In some cases, aspects of operation 356 are performed by a region identifier, such as region identifier 110 of FIG. 1.

[0070] Flow proceeds to operation 358, where a set of chromophores is generated for the control region determined in operation 356. Aspects of operation 358 may be similar to those previously discussed with respect to operation 308 of method 300 of FIG. 3A, and therefore need not be described in detail again. For example, aspects of operation 358 may be performed by a chromophore processor, such as chromophore processor 112 of FIG. 1. Operation 358 may include determining a set of chromophore concentrations to contribute to or account for the coloration exhibited in the image data corresponding to the control region, according to any of the various techniques described above.

[0071] At operation 360, an estimated thickness of the image data corresponding to the region of interest is generated based on the set of chromophore concentrations generated at operation 358. Aspects of operation 360 may be similar to those previously discussed with respect to operations 310 and / or 312 of method 300 of FIG. 3A and therefore need not be described in detail again. In some embodiments, multiple thickness estimates are generated (e.g., for multiple pixels or other subparts within the region of interest). In such embodiments, a thickness map including the thickness estimates may be generated accordingly, thereby indicating a thickness gradient within the region of interest. As another example, the resulting estimate includes an indication of whether the region of interest exhibits abnormal skin thickness, which may thus be a binary indication.

[0072] Moving to operation 362, an indication of the estimated thickness(es) of the region of interest is provided. For example, the indication may be provided to a computing device, such as computing device 104 of FIG. 1 . Alternatively or additionally, an indication is provided as to whether the region of interest exhibits abnormal skin thickness. In other examples, the indication is stored in a data store for subsequent evaluation, which may be the case when analyzing the progression of the corresponding condition over time. As a further example, the indication may be provided for subsequent processing to generate a symptom severity metric according to aspects described herein, which may be generated by a symptom determination engine (e.g., symptom determination engine 116 of FIG. 1 ) performing aspects of method 400 of FIG. 4A , which will be described in more detail below. Method 350 ends at operation 362. As another example, the indication may be provided for subsequent processing to determine whether a skin condition is present according to aspects described herein, such as may be generated by performing aspects of method 450 of FIG. 4B .

[0073] 4A illustrates an overview of an example method 400 for determining symptom severity based on estimated thickness and estimated roughness according to aspects described herein. In an embodiment, aspects of method 400 are performed by a data processing platform, such as data processing platform 102 of FIG. 1 .

[0074] Method 400 begins with operation 402, in which image data is acquired. For example, the image data may be acquired from a data store or from an image capture device (e.g., image capture device 118 of computing device 104 in FIG. 1 ), among other examples. The image data includes at least a portion of a user's body, such as one or more areas affected by a skin / nail condition. Aspects of operation 402 may be similar to those previously discussed with respect to operations 302 and 352 of methods 300 and 350, respectively, and therefore need not be described in detail again.

[0075] In operation 404, an estimated thickness is generated for the region of interest of the image data acquired in operation 402. Operation 404 may include performing aspects of methods 300 and / or 350 discussed above with respect to Figures 3A and 3B, respectively.

[0076] Flow proceeds to operation 406, where an estimated surface roughness is generated. Aspects of operation 406 may be performed by a roughness determiner, such as roughness determiner 115 of FIG. 1. As described above, any of a variety of techniques may be used to generate a surface roughness metric. For example, in operation 406, a grayscale representation of the region of interest (which may be generated, for example, by an image preprocessor, such as image preprocessor 108) is processed to generate one or more strips therein, whereby light / dark variations therein may be used to generate a surface roughness metric in response.

[0077] Operation 406 is illustrated with a dashed box to indicate that operation 406 may be omitted in some embodiments. For example, the symptom severity metric may be generated based on the estimated thickness and estimated surface roughness (e.g., in cases where operations 404 and 406 are performed), or based on the estimated thickness (e.g., in cases where operation 406 is omitted), among other embodiments.

[0078] Moving to operation 408, a symptom severity corresponding to the region of interest is determined based on the estimated thickness generated in operation 404 and, optionally, the estimated surface roughness generated in operation 406. Aspects of operation 408 may be performed by a symptom determination engine, such as symptom determination engine 116 of Figure 1. Aspects of operation 408 may be similar to those previously discussed with respect to operation 314 of Figure 3A, and therefore need not be described in detail again below.

[0079] For example, the severity metric may include a weighted average of the estimated thickness and surface roughness. In other examples, a machine learning model is used to generate a classification accordingly based on the estimated thickness and / or surface roughness. Thus, it will be appreciated that any of a variety of techniques may be used to generate a symptom severity metric based on the estimated thickness and / or surface roughness in accordance with aspects described herein.

[0080] At operation 410, an indication of the symptom severity metric is provided. For example, the indication may be provided to a computing device, such as computing device 104 of FIG. 1. In other examples, the indication is stored in a data store for subsequent evaluation, which may be the case when analyzing the progression of the corresponding symptom over time. Method 400 ends at operation 410.

[0081] 4B illustrates an overview of an example method 450 for determining whether a skin condition is present according to aspects described herein. In an embodiment, aspects of method 400 are performed by a data processing platform, such as data processing platform 102 of FIG. 1 .

[0082] Method 450 begins with operation 452, in which image data is acquired. For example, the image data may be acquired from a data store or from an image capture device (e.g., image capture device 118 of computing device 104 of FIG. 1 ), among other examples. The image data includes at least a portion of a user's body, such as one or more areas affected by a skin / nail condition. Aspects of operation 452 may be similar to those previously discussed with respect to operations 302, 352, and / or 402 of methods 300, 350, and 400, respectively, and therefore need not be described in detail again.

[0083] At operation 454, it is determined whether the region of interest in the image data exhibits an abnormal thickness. Aspects of operation 454 may be performed by a thickness evaluator, such as thickness evaluator 114 discussed above with respect to FIG. 1. As noted above, any of a variety of techniques may be used to determine whether the region of interest exhibits an abnormal thickness. As one example, one or more of the equations described above are used to process the image data corresponding to the region of interest and the control region.

[0084] Flow proceeds to operation 456, where an estimated surface roughness is generated. Aspects of operation 406 may be performed by a roughness determiner, such as roughness determiner 115 of FIG. 1. As described above, any of a variety of techniques may be used to generate a surface roughness metric. For example, in operation 406, a grayscale representation of the region of interest (which may be generated, for example, by an image preprocessor, such as image preprocessor 108) is processed to generate one or more strips therein, whereby light / dark variations therein may be used to generate a surface roughness metric in response.

[0085] At decision 458, it is determined whether the skin thickness is abnormal. For example, this determination may include determining whether the processing performed at operation 454 produced a true assessment processing result. If it is determined that the skin thickness is not abnormal, flow branches "NO" to decision 460, where it is determined whether the skin texture is abnormal. If it is determined that the skin texture is not abnormal, flow branches "NO" to operation 462, where an indication is provided (e.g., to a computing device, such as computing device 104 of FIG. 1 ) indicating that a skin condition is unlikely to be present. It will be appreciated that any of a variety of additional or alternative operations may be performed, such as storing the indication in association with the image data, thereby enabling past or retrospective analysis of the individual's skin. Method 450 ends at operation 462.

[0086] Returning to decision 460, conversely, if it is determined that the skin texture is abnormal, flow branches "yes" to operation 464, where instructions to consult a professional are provided (e.g., to a computing device, such as computing device 104 of FIG. 1). Thus, method 450 may produce an indeterminate outcome if the skin thickness is not abnormal (e.g., as a result of the "no" branch at decision 458) and the skin exhibits abnormal texture (e.g., the "yes" branch at decision 460). As with operation 462, any of a variety of additional or alternative processes may be performed at operation 464. Method 450 ends at operation 464.

[0087] Returning to decision 458, conversely, if the skin thickness is determined to be abnormal, flow branches "yes" to decision 466, where it is determined whether the skin texture is abnormal. Aspects of decision 466 may be similar to those previously discussed with respect to operation 460, and therefore need not be repeated in detail. Thus, if the skin texture is determined to be not abnormal, flow branches "no" to previously discussed operation 464. Thus, method 450 may produce an indeterminate outcome if the skin thickness is abnormal (e.g., as a result of the "yes" branch at decision 458) and the skin does not exhibit abnormal texture (e.g., the "no" branch at decision 466).

[0088] Returning to decision 466, conversely, if the skin texture is determined to be abnormal, flow branches "yes" to operation 468, where an indication is provided (e.g., to a computing device, such as computing device 104 of FIG. 1) indicating that a skin condition may be present. It will be appreciated that any of a variety of additional or alternative operations may be performed to store the indication, for example, in association with the image data, thereby enabling historical or retrospective analysis of the individual's skin. Method 450 ends at operation 466.

[0089] 5 illustrates one example of a suitable operating environment 500 in which one or more of the present embodiments may be implemented. This is merely one example of a suitable operating environment and is not intended to suggest any limitation as to the scope of use or functionality. Other known computing systems, environments, and / or configurations that may be suitable for use include, but are not limited to, personal computers, server computers, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, programmable consumer electronic devices such as smartphones, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.

[0090] In its most basic configuration, an operating environment 500 may typically include at least one processing unit 502 and memory 504. Depending on the exact configuration and type of computing device, the memory 504 (which stores, among other things, APIs, programs, and the like, and / or other components or instructions for implementing or executing the systems and methods disclosed herein) may be volatile (e.g., RAM), non-volatile (e.g., ROM, flash memory, and the like), or a combination of the two. This most basic configuration is illustrated by dashed line 506 in FIG. 5. Furthermore, the environment 500 may also include storage devices (removable 508 and non-removable 510), including, but not limited to, magnetic or optical disks or tape. Similarly, the environment 500 may also include input device(s) 514, such as a keyboard, mouse, pen, voice input, etc., and / or output device(s) 516, such as a display, speakers, printer, etc. The environment may also include one or more communications connections 512, such as a LAN, WAN, point-to-point, etc.

[0091] The operating environment 500 may include at least some form of computer-readable media. Computer-readable media may be any available media that can be accessed by the processing unit 502 or other devices comprising the operating environment. For example, computer-readable media may include computer storage media and communication media. Computer storage media may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media may include RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or other non-transitory media that can be used to store the desired information. Computer storage media need not include communication media.

[0092] Communication media may embody computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term "modulated data signal" may mean a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. For example, communication media may include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Additionally, combinations of any of the above are also intended to be included within the scope of computer-readable media.

[0093] The operating environment 500 may be a single computer operating in a networked environment using logical connections to one or more remote computers. The remote computers may be personal computers, servers, routers, network PCs, peer devices, or other common network nodes, and typically include many or all of the elements listed above, as well as other elements not listed above. The logical connections may include any method supported by an available communications medium. Such networked environments are commonplace in offices, enterprise-wide computer networks, intranets, and the Internet.

[0094] Various aspects described herein may be employed using software, hardware, or a combination of software and hardware to implement and perform the systems and methods disclosed herein. Although particular devices have been referenced throughout this disclosure as performing particular functions, those skilled in the art will recognize that such devices are provided for illustrative purposes and that other devices may be employed to perform the functions disclosed herein without departing from the scope of the present disclosure.

[0095] As mentioned above, a number of program modules and data files may be stored in system memory 504. While executing on processing unit 502, the program modules (e.g., applications, input / output (I / O) management, and other utilities) may perform processes that include one or more of the steps of methods of operation described herein, such as, but not limited to, the methods illustrated in Figures 2, 3A-3B, or 4A-4B.

[0096] Furthermore, embodiments of the present invention may be implemented in electrical circuits including discrete electronic elements, packaged or integrated electronic chips containing logic gates, circuits utilizing a microprocessor, or on a single chip containing electronic elements or a microprocessor. For example, embodiments of the present invention may be implemented via a system-on-chip (SOC), which may integrate each or many of the components illustrated in FIG. 5 onto a single integrated circuit. Such an SOC device may include one or more processing units, graphics units, communications units, system virtualization units, and various application functions, all integrated (or "burned") onto the chip substrate as a single integrated circuit. When operating via an SOC, the functionality described herein may be operated via application-specific logic integrated with other components of the operating environment 500 on a single integrated circuit (chip). Embodiments of the present disclosure may also be implemented using other technologies capable of performing logical operations such as, for example, AND, OR, and NOT, including, but not limited to, mechanical, optical, fluidic, and quantum technologies. Furthermore, embodiments of the present invention may be implemented within a general-purpose computer or other circuit or system.

[0097] For example, aspects of the present disclosure are described above with reference to block diagrams and / or operational illustrations of methods, systems, and computer program products according to aspects of the present disclosure. The functions / acts described in the blocks may occur out of the order shown in the flowcharts. For example, two blocks shown in succession may in fact be executed substantially concurrently, or in some cases, the blocks may be executed in the reverse order, depending on the functions / acts involved.

[0098] The description and illustrations of one or more aspects provided in this application are not intended to limit or restrict the scope of the claimed disclosure in any way. The aspects, examples, and details provided in this application are believed to be sufficient to transfer ownership of the invention and to enable others to make and use the best mode of the claimed disclosure. The claimed disclosure is not to be construed as limited to any aspect, example, or detail provided in this application. Various features (both structural and methodological), whether shown and described in combination or individually, are intended to be selectively included or omitted to produce an embodiment with a particular set of features. Given the description and illustrations provided in this application, those skilled in the art will be able to envision variations, modifications, and alternatives that are within the spirit of the broader aspects of the general inventive concepts embodied in this application and do not depart from the broader scope of the claimed disclosure.

Claims

1. 1. A system comprising: at least one processor; a memory storing instructions that, when executed by the at least one processor, cause the system to perform a set of operations, the set of operations including: obtaining image data representing a control area of ​​the user's skin and image data of the area of ​​interest of the skin; generating a set of chromophore concentrations based on the image data representing the control region; generating an estimated skin thickness of the skin within the region of interest based on the set of chromophore concentrations and the image data of the region of interest; and providing an indication of the estimated skin thickness.

2. The set of operations further includes pre-processing the image data representing the control area to generate a set of wavelength band intensity values ​​for the control area; The system of claim 1 , wherein the set of chromophore concentrations is generated based on the set of wavelength band intensity values.

3. The estimated skin thickness is generating a predicted set of wavelength band intensity values ​​based on said set of chromophore concentrations; calculating a first value based on a difference between the expected set of wavelength band intensity values ​​and the set of wavelength band intensity values ​​of the control region; calculating a second value based on a difference between the set of wavelength band intensity values ​​of the region of interest and the set of wavelength band intensity values ​​of the control region; and performing a comparison between the first value and the second value.

4. 4. The system of claim 3, wherein the first value is calculated by multiplying the difference between the expected set of wavelength band intensity values ​​and the set of wavelength band intensity values ​​of the control region by an adjustment factor.

5. The system of any one of claims 3 to 4, wherein the estimated skin thickness is generated by calculating the difference between the first value and the second value.

6. The set of operations further includes determining a skin condition severity based on the estimated skin thickness; The system of any one of claims 1 to 5, wherein the provided instructions further comprise the determined skin condition severity.

7. The set of operations further includes generating an estimated skin surface roughness for the region of interest; The system of claim 6 , wherein the skin condition severity is further determined based on the estimated skin surface roughness.

8. The system of any one of claims 1 to 7, wherein the set of operations further comprises receiving user input comprising an indication of the control region and the region of interest.

9. The system of any one of claims 1 to 7, wherein the set of operations further comprises automatically determining the control region and the region of interest.

10. The system of any one of claims 1 to 9, wherein at least one of the image data representing the control region or the image data of the region of interest is obtained from an image capture device of the system.

11. The system of any one of claims 1 to 10, wherein at least one of the image data representing the control region or the image data of the region of interest is obtained from a data store.

12. 12. The system of claim 1, wherein the set of chromophore concentrations is generated by processing the image data representing the control region using an estimation matrix that outputs estimated concentrations of one or more chromophores.

13. The set of chromophore concentrations is melanin, bilirubin, oxygenated blood, and 13. The system of any one of claims 1 to 12, comprising a concentration of one or more chromophores selected from the group of chromophores consisting of deoxygenated blood.

14. The system of any preceding claim, wherein the indication of the estimated skin thickness comprises a thickness of the area of ​​interest relative to the control area.

15. The system of any preceding claim, wherein the image data representative of the control area comprises data derived from an image of the control area of ​​the skin of the user.

16. The system of any one of claims 1 to 14, wherein the image data representing the control area includes data derived from image data of a control area of ​​the skin of another user different from the user.

17. 1. A method for generating a skin condition severity based on image data, the method comprising: acquiring image data representative of a control area of ​​the user's skin; acquiring image data of the region of interest of the skin; generating a set of chromophore concentrations based on the image data representing the control region; generating an estimated skin thickness of the skin within the region of interest based on the set of chromophore concentrations and the image data of the region of interest; and providing an indication of said estimated skin thickness.

18. The method further includes preprocessing the image data representing the control area to generate a set of wavelength band intensity values ​​for the control area; The method of claim 17 , wherein the set of chromophore concentrations is generated based on the set of wavelength band intensity values.

19. The estimated skin thickness is generating a predicted set of wavelength band intensity values ​​based on said set of chromophore concentrations; calculating a first value based on a difference between the expected set of wavelength band intensity values ​​and the set of wavelength band intensity values ​​of the control region; calculating a second value based on a difference between the set of wavelength band intensity values ​​of the region of interest and the set of wavelength band intensity values ​​of the control region; and performing a comparison between the first value and the second value.

20. 20. The method of claim 19, wherein the first value is calculated by multiplying the difference between the expected set of wavelength band intensity values ​​and the set of wavelength band intensity values ​​of the control region by an adjustment factor.

21. The method of any one of claims 19 to 20, wherein the estimated skin thickness is generated by calculating the difference between the first value and the second value.

22. The method further comprising determining a skin symptom severity based on the estimated skin thickness; The method of any one of claims 17 to 21, wherein the provided instructions further comprise the determined skin condition severity.

23. The method further includes generating an estimated skin surface roughness for the region of interest; 23. The method of claim 22, wherein the skin condition severity is further determined based on the estimated skin surface roughness.

24. The method of any one of claims 17 to 23, further comprising receiving user input comprising an indication of the control region and the region of interest.

25. The method of any one of claims 17 to 24, further comprising automatically determining the control region and the region of interest.

26. 26. The method of any of claims 17 to 25, wherein the set of chromophore concentrations is generated by processing the image data of the control region using an estimation function that outputs an estimated concentration of one or more chromophores.

27. A method according to any of claims 17 to 26, wherein the image data representative of the control area comprises data derived from an image of the control area of ​​the skin of the user.

28. A method according to any one of claims 17 to 26, wherein the image data representing the control area comprises data derived from image data of a control area of ​​the skin of another user different from the user.

Images