Approaches to determining a target material via identification and analysis of spectral information using non-occluded illumination

Abstract

Introduced here are approaches that allow for usable, real-world reflectance spectroscopy to be achieved using a mobile device – like a smartphone – without any additional external hardware, in the presence of ambient light. These approaches enable spectroscopic measurements of material properties, as well as the mapping and imaging of material properties (and importantly, the ability to track changes in material properties over time in a quantifiable manner). The application of these approaches are widespread, including precise determination of color; identification of skin and hair chromophore and condition; identification of material; identification of changes in material state (e.g., due to sunlight, moisture, aging, use, etc.); detection of counterfeit articles; verification of living bodies (e.g., for security purposes); authentication of documentation; and agriculture, for example, to identify plant materials and establish condition.

Description

APPROACHES TO DETERMINING A TARGET MATERIAL VIA IDENTIFICATION AND ANALYSIS OF SPECTRAL INFORMATION USING NON-OCCLUDED ILLUMINATIONCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to US Provisional Application No. 63 / 714,858, titled “Method For Determining A Target Material Via Identification Of Spectra Using Non-Occluded Illumination” and filed on October 31, 2024, and US Provisional Application No. 63 / 807,065, titled “Method For Determining A Target Material Via Identification Of Spectra Using Non-Occluded Illumination” and filed on May 16, 2025, each of which is hereby incorporated by reference in its entirety.TECHNICAL FIELD

[0002] Various embodiments concern computer programs and associated computer-implemented approaches to characterizing a target material via analysis of spectral information.BACKGROUND

[0003] Reflectance spectroscopy is a well-established analytical technique for identifying or characterizing a material based on the light it reflects when illuminated by a known source. The technique relies on measuring the spectrum of reflected light and then comparing it to the spectrum of the illumination source to determine wavelengthdependent reflectivity. A known, stable, and sufficiently intense illumination source is essential to overcome system noise - including detector noise, stray light, and electronic interference. Performing measurements in the absence of ambient lighting simplifies this process by eliminating the need to account for uncontrolled or varying external light sources. While several approaches have been proposed or developed for subtracting ambient light contributions, such corrections typically introduce additional noise and uncertainty.

[0004] Using these principles, materials can be identified through their specific reflectance signatures within a defined spectral range. The spectral resolution - that is,the ability to distinguish small wavelength differences - is determined by the combined bandwidths of the illumination source, the reflected light collection optics, and the detector. Broadly speaking, three classes of reflectance spectroscopy systems exist:• Systems that employ broadband illumination with narrowband - for example, through scanning or multi-channel - detection;• Systems that use narrowband or tunable illumination with broadband detection; and• Systems that use multi-channel or scanning illumination and detection in combination.

[0005] Higher spectral resolution and improved detector sensitivity lead to more accurate material identification, particularly for materials with subtle or overlapping spectral features.BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1A describes a reflectance spectrum for the skin of a living body.

[0007] Figure 1 B describes the reflectance spectrums of skin with different material properties.

[0008] Figure 2 describes a spectral model for a target configured to reflect different parameter configurations.

[0009] Figure 3 illustrates the addition of a parameter into an spectral model for a target.

[0010] Figure 4 illustrates the determination of one or more properties of a target by fitting a spectral model developed for the target to a measured curve of a sample of the target.

[0011] Figure 5 illustrates a process for differentially illuminating an object for generation of a reflectance spectrum of the object.

[0012] Figure 6 illustrates a process for characterizing a material property of a target.

[0013] Figure 7 illustrates an example of a computing device with an illuminant and a display for illuminating an object of interest and a number of image sensors for generating digital images in conjunction with illumination events performed by the illuminant or display.

[0014] Figure 8 illustrates an example set of spectral outputs of a known illuminant.

[0015] Figure 9 illustrates an example set of spectral responses of an image sensor.

[0016] Figure 10 includes a schematic diagram of the approach for establishing the spectral fingerprint of an object.

[0017] Figure 11 describes a process to determine a target property based on an analysis of values output by a multi-channel image sensor.

[0018] Figure 12 illustrates a set of training reflectance spectrums that could be utilized in the process of determining a target property based on an analysis of values output by a multi-channel image sensor.

[0019] Figure 13 describes a process to determine a change in a property of an object over time.

[0020] Figure 14 describes a process for determining a change in a property, specifically a property of skin, by an application implemented on a computing device.

[0021] Figure 15 describes a process for carrying out ABDCE evaluation for skin melanoma detection.

[0022] Figure 16 illustrates a network environment that includes a characterization module executing on a computing device. Figure 16 illustrates a network environment 1600 that includes a characterization module 1604 executing on a computing device 1602.

[0023] Figure 17 is a block diagram illustrating an example of a processing system that is capable of implementing the operations described herein.DETAILED DESCRIPTION

[0024] Transforming a mobile device - like a smartphone or tablet computer - into a low-cost, widely available reflectance spectroscope has long been an attractive goal. Such a device could enable applications ranging from consumer material identification to medical diagnostics and environmental monitoring. However, previous attempts have typically relied on external hardware modules - such as diffraction gratings, lenses, or light-emitting diode (LED) arrays - to achieve the required optical and spectral performance. When efforts have been made to perform reflectance spectroscopy using only the built-in components of smartphones (e.g., via mobile applications executing thereon), the results have been unreliable.

[0025] This unreliability stems from several fundamental limitations of smartphone hardware and operating conditions.

[0026] First, smartphone cameras are not spectrally calibrated. Their complementary metal-oxide-semiconductor (CMOS) sensors are optimized for color imaging using broad red-green-blue (RGB) filters, not for narrowband spectral discrimination. This means they cannot distinguish fine wavelength variations that define a material’s reflectance spectrum.

[0027] Second, the illumination sources in smartphones - like the LEDs used in backlighting displays or flash units - have irregular and device-specific spectral profiles that vary with temperature, age, and power level. Without a known, stable reference spectrum, it becomes impossible to compute accurate reflectance ratios.

[0028] Third, smartphones lack optical isolation and geometric control.Measurements are taken in uncontrolled environments with arbitrary angles between the light source, sample, and detector, leading to unpredictable scattering and shadowing effects. Ambient light - often from mixed, spectrally complex sources such as sunlight, fluorescent lighting, or displays - further contaminates the signal. Although some computational methods attempt to estimate and subtract ambient contributions, these approaches struggle when the spectral composition of ambient light is unknown or spatially non-uniform.

[0029] In professional-grade reflectance spectrometers, these challenges are mitigated through calibrated light sources, well-characterized detectors, narrowbandoptical filters, and controlled geometries that eliminate stray light and reflections. In contrast, smartphones lack these dedicated components, making it historically difficult -if not impossible - to achieve the signal-to-noise ratio, spectral resolution, and reproducibility required for accurate reflectance spectroscopy in real-world conditions.

[0030] Introduced here, therefore, are approaches that allow for usable, real-world reflectance spectroscopy to be achieved using a mobile device - like a smartphone -without any additional external hardware, in the presence of ambient light. As further discussed below, these approaches enable spectroscopic measurements of material properties, as well as the mapping and imaging of material properties (and importantly, the ability to track changes in material properties over time in a quantifiable manner). The application of these approaches are widespread, including precise determination of color; identification of skin and hair chromophore and condition; identification of material; identification of changes in material state (e.g., due to sunlight, moisture, aging, use, etc.); detection of counterfeit articles; verification of living bodies (e.g., for security purposes); authentication of documentation; and agriculture, for example, to identify plant materials and establish condition.Terminology

[0031] References to “an embodiment” or “some embodiments” mean that the feature being described is included in at least one embodiment. Occurrences of such phrases do not necessarily refer to the same embodiment, nor are these occurrences necessarily referring to alternative embodiments that are mutually exclusive of each other.

[0032] The term “based on” is to be construed in an inclusive sense rather than an exclusive sense. That is, in the sense of “including but not limited to.” Accordingly, the term “based on” is meant to be interpreted as “based at least in part on” unless otherwise noted.

[0033] The terms “connected,” “coupled,” and variants thereof are intended to cover any connection or coupling between two or more components, either direct or indirect. For example, components may be electrically or communicatively coupled to each other despite not sharing a physical connection.

[0034] The term “module” may be used to refer to a self-contained unit of functionality that is implemented in software, firmware, hardware, or any combination thereof. A module typically operates by receiving one or more inputs, processing those inputs through defined operations or logic, and then producing one or more outputs. Modules are often designed with standardized interfaces to enable integration, reuse, and independent development or maintenance within complex systems. However, modules can also be designed with specialized interfaces that allow them to occupy a specific position within a processing pipeline, where each module takes, as input, the output of a preceding module and supplies its own output as input to a succeeding module, enabling a structured and sequential flow of data or operations.Overview of Spectral Modeling

[0035] A “reflectance spectrum” (also called a “reflectance curve,” “reflectivity spectrum,” or “reflectivity curve”) describes how strongly a material reflects light at different wavelengths across the electromagnetic spectrum. Consider, for example, a scenario where a white light - which contains all visible wavelengths - is shone on a material. Some wavelengths will be reflected more strongly than others, depending on the material’s surface properties and composition. If you measure and plot the ratio of reflected light intensity to incident light intensity as a function of wavelength, the resulting plot is the reflectance spectrum. Mathematically, the reflectance spectrum is expressed as:R(λ) = Ireflected(λ)R W~ ncidentW ’Eq'1where / ?( ) is reflectance at wavelength A, frefiectedW is the intensity of reflected light, and ImcidentW is the intensity of the illumination source at that same wavelength.

[0036] The reflectance spectrum of a material is unique to that material and can be used in identifying its composition, color, and physical properties.

[0037] For example, sampled areas of skin across different areas of a living body -or between different living bodies - produce unique reflectance spectrums based on an integration of the reflectance spectra of different chromophores (e.g. melanin, oxygenated blood, de-oxygenated blood, total blood volume, and bilirubin), water, sebum content, and the like within the sampled area. Figure 1A includes an example ofa reflectance spectrum for the skin of a living body. As further discussed below, the reflectance spectrum shown in Figure 1A has been generated by a spectral model designed and / or trained to produce reflectance spectrums for skin. Figure 1 B, meanwhile, includes three examples of reflectance spectrums of skin with different material properties. The uppermost reflectance spectrum 102 is that of sampled skin with zero melanin, zero blood volume, and zero blood oxygenation. The middle reflectance spectrum 104 is that of sampled skin with substantial blood volume, but leaving the blood with no oxygenation. The lowermost reflectance spectrum 106 is that of sampled skin with substantial blood oxygenation. Because photon penetration depth into tissue is also a function of wavelength, scattering and absorption properties of various layers of the skin will impact the integrated spectra.

[0038] While the reflectance spectra shown in Figures 1 B are examples of gross changes in properties of the skin, the same methodology can be used to measure - and visually illustrate - very fine changes, quantify the chromophore variation, and quantify additional chromophores that might otherwise have minor impacts on appearance. As further discussed below, the spectral model developed for the skin allows specific chromophore variations (e.g., blood volume, blood oxygenation, water, melanin, etc.) to be integrated into the reflectance spectra, as shown in Figures 1A-B. The ranges across which these chromophores can be varied may be predetermined to take into account the range of each chromophore that is possible within a normal distribution of the human population.

[0039] Those skilled in the art will recognize this approach - namely, developing a spectral model for a target material (also called a “target surface,” “target object,” or simply “target”) and then allowing for chromophore variation - could be extended to other applications than the skin. For example, the reflectance spectrum of plant leaves is affected by their chromophores (e.g. chlorophyll, anthocyanins, etc.), water content, cellular structure, and other factors. Healthy leaves absorb more strongly in the blue and red regions of the visible spectrum (due to chlorophyll), while they reflect most green wavelengths. When a leaf is stressed, for example, due to disease or nutrient deficiency, these spectral features change - such as a reduction in red wavelength absorption or increased reflectance in the yellow and / or brown regions of the visiblespectrum -allowing for precise identification of leaf health and species through spectral analysis.

[0040] In order to determine the chemical properties of a target from a reflectance spectrum, it is helpful to build a spectral model (or simply “model”) of the target. Such a model describes the reflectivity of its constituent material(s), taking into account the contributions or concentrations of a number of elements. It is worth noting that the spectral model of a target may not be a linear combination of the spectral reflectances of its constituent materials. Various parameters may affect the shape and character of the reflectance spectrum in arbitrary ways. Because of the complexities of parameter effects on reflectance spectra there are a number of ways to build a spectral model for a target material.

[0041] In some embodiments, the spectral model of a target is constructed by leveraging one or more existing reflectance spectra. These models are typically derived from experimental data or scientific literature and describe the reflectance properties of common chromophores or constituent materials. Existing reflectance spectra may serve as basis functions, and the spectral model is generated by combining them. For example, a spectral model that is developed for stainless steel could be based on a reflectance spectrum - or multiple reflectance spectra - obtained from scientific literature, as significant experimentation of stainless steel has taken place.

[0042] In other embodiments, the spectral model of a target is constructed through direct collection of reflectance data from at least one representative sample (also called a “representative example”) of the target through spectroscopy. The representative sample is illuminated with a controlled light source, such as a broadband white light or a series of narrow-band LEDs, and the reflected light is measured by a spectrometer or sensor array, capturing reflectance values across a wavelength range of interest (usually about 300-750 nanometers). These measurements produce a high-resolution reflectance spectrum that is unique to the representative sample. The model can then be built by fitting mathematical functions to this measured data and associating observed spectral features with known chromophores or optical elements.

[0043] In other embodiments, a spectral model is developed for a target by first acquiring multiple spectral responses of a multi-channel image sensor that is used toimage a representative sample of the target, where each of the spectral responses corresponds to a different sensor channel of the multi-channel image sensor. Multiple illumination spectra are obtained from a multi-channel light source, with each spectrum corresponding to a distinct illumination channel of the multi-channel light source.Subsequently, multiple spectral fingerprints are produced by performing element-wise multiplication of the respective spectral responses and illumination spectra for each possible sensor-illumination channel pair. The collection of these spectral fingerprints enables quantitative mapping of sensor output to material properties defined in a parametric spectral model, thereby facilitating the determination of parameters of the target through systematic analysis of the multi-channel response.

[0044] As further discussed below, one of the benefits of developing a spectral model for a target is the ability to make the spectral model variable based on different parameters. Figure 2, for example, includes two different parameter configurations of a four-parameter spectral model 200 developed for skin. Here, the four parameters are blood volume, blood oxygenation, water content, and melanin content. Usually the spectral model 200 is independently manipulated by a computer program called a “characterization module” as further discussed below. However, a user may also be able to manipulate the spectral model 200 through an interface, for example, that is generated by the characterization module.

[0045] To allow for easier input, sliders 202 may be incorporated into the interface through which the user interacts with the spectral model 200. The number of sliders 202 may correspond to the number of adjustable parameters, allowing the user to interactively and intuitively adjust the parameters corresponding to key chromophores in the combined spectral function. As the user moves a slider, the concentration or proportion of the selected chromophore is increased or decreased, which directly modifies the shape and features of the overall reflectance spectrum rendered on the interface. Said another way, through the sliders 202, the user may be able to produce a theoretical reflectance spectrum to reflect a unique combination of relative blood volume 204, degree of blood oxygenation 206, water content 208, and quantity of melanin 210. Referring again to Figure 2, the upper reflectance spectrum 212 has a blood volume 204 of 0.010, blood oxygenation 206 of 0.500, water content 208 of 0.650, and melaninof 0.016, blood oxygenation 206 of 0.150, water content 208 of 0.737, and melanin 210 of 0.027.

[0046] In embodiments where the characterization module is responsible for manipulating the values of the parameters, the sliders 202 are not needed - in fact, the spectral model 200 may not be presented on an interface at all - but serve as a helpful means for conceptualizing how the characterization module might iterate each parameter across its corresponding range of values to produce different reflectance spectra.

[0047] Either way, this methodology can be used to determine, for a given reflectance spectrum and a given target, the chromophore values that are needed to achieve the shape of that curve.

[0048] In some embodiments, the spectral model of a target consists only of elements that meaningfully impact the reflectance spectra. Said another way, only elements for which inclusion of the corresponding reflectance spectra has a statistically significant impact on the spectral model may be included in the spectral model.Elements for which the impact is not statistically significant - for example, for which the impact is below a change threshold - can be held constant. Accordingly, the number of variables that are used in constructing the spectral model is highly dependent on the material application. Methods for constructing spectral models can vary depending on the type or subset of targets being analyzed.

[0049] In some embodiments, the maximum and minimum values of the sliders 202 are predetermined to take into account the range of each chromophore or other parameter that is possible within the normal distribution of the target material. In Figure 2, the chromophore ranges are based on the normal distribution of the human population. Lessening the total range of values has several notable benefits. First, limiting the range allows for lower computational cost without a meaningful impact on accuracy. Second, limiting the range allows for improved determination speed, which may be especially important if this approach is used to analyze digital images - or at least parts of digital images - that are representative of frames of a video in near real time.

[0050] In some embodiments, additional chromophores of a material may be added to an existing spectral model. Additional chromophores may be added both within the visible spectrum and beyond the visible spectrum into the near-ultraviolet (NUV) spectrum or near-infrared (NIR) spectrum, depending on the range of the illumination spectrum and detection spectrum. The addition of more chromophores to an existing spectral model may provide further information about the characteristics of the target, leading to a wider range of target property determinations.

[0051] Figure 3 illustrates the addition of bilirubin, a chromophore of skin in the visible domain, to the spectral model of skin. Bilirubin is produced during the normal breakdown of red blood cells in the body. High levels of bilirubin are indicative of liver or bile duct issues in adults. In newborns, where the liver cannot break down the bilirubin at the rate it is produced, high bilirubin concentrations (commonly diagnosed as jaundice) can result. In newborns, this can lead to major developmental issues and even death. With approximately 60 percent of term and 80 percent of preterm newborns developing jaundice in the first week after birth, this is a serious condition that needs monitoring multiple times daily. The reflectance spectrum of bilirubin 302 is well-known within the literature and illustrated in the uppermost graph of Figure 3.

[0052] To incorporate the bilirubin reflectance spectrum into the spectral model of skin, an additional parameter for bilirubin is added to the existing spectral model. This new parameter functions as a coefficient representing the proportional concentration of bilirubin within the skin, similar to how the relative concentration of melanin or blood affects the existing spectral model developed for the skin. The spectral model becomes a sum of its constituent chromophores, with the bilirubin component scaled according to its modeled abundance.

[0053] In Figure 3, two additional graphs are shown. The middle graph shows the spectral model of the skin with zero percent bilirubin included. When the bilirubin parameter 304 is set to zero, the skin reflectance spectrum is not influenced by the bilirubin reflectance spectrum at all. However, as the parameter is increased, the reflectance curve shifts, increasingly resembling the bilirubin reflectance spectrum. The lowermost graph shows the spectral model of the skin with bilirubin set to a value of 0.300. With one-hundred percent bilirubin included, the skin reflectance spectrum moreclosely resembles the bilirubin reflectance spectrum, particularly around 450-nm wavelengths. With the addition of the bilirubin parameter into the spectral model of skin, the spectral model may be able to simulate varying degrees of bilirubin, leading to more sensitive detection and quantification of bilirubin-related conditions using spectroscopic analysis.

[0054] A similar method may be used to incorporate other parameters into a spectral model, whether that existing model is for skin or for another target material. In some embodiments, additional parameters may be added to a spectral model similar to the methods described above to build a spectral model: through existing spectral models or through direct data collection of reflectance data.

[0055] Figure 4 illustrates the determination of one or more properties of a target by fitting a spectral model developed for the target to a measured curve of a sample of the target. The method of fitting a spectral model to a measured curve may be applied to a number of materials and the spectral model may consist of a number of inputs depending on the target. Figure 4 utilizes the spectral model of skin similar to that of Figure 1 with the parameters of relative blood volume 404, degree of blood oxygenation 406, water content 408, and quantity of melanin 410.

[0056] For example, a sample of skin may have the measured reflectance spectrum 412. A reflectance spectrum may be measured through a number of techniques.Reflectivity of a material may be measured using reflective spectroscopy systems including, but not limited to, broadband illumination source and narrow bandwidth scanning (or multi-channel) detectors, narrow bandwidth scanning (or multi-channel) illumination source and a broadband detector, or a combination of multi-channel (or scanning) illumination and multi-channel (or scanning) detection.

[0057] In some embodiments, the measured reflectance spectrum 412 of a sample of skin is superimposed into the spectral model of skin to determine the properties of the sample. Assuming that the measured sample of skin fits within the parameters of the spectral model, the optimal model parameters that minimize the difference between the measured reflectance spectrum 412 and modeled curve 414 are found. Figure 4 illustrates the adjustment of the parameter sliders 402 to minimize the difference between the measured curve 412 and model curve 414. For the purpose of illustration,a single parameter - namely, the degree of blood oxygenation 406 - is decreased to minimize the difference.

[0058] Those skilled in the art will appreciate that any number of numerical methods could be used to find the best set of parameters to match the measured reflectivity curve. In some embodiments, the difference between the measured and modeled reflectance values at each wavelength is squared and summed across all wavelengths to determine a total error or sum of squares. In some embodiments, the error is minimized by adjusting the model parameters manually (e.g., using the sliders 402) or automatically with an iterative algorithm. Other methods may include, but are not limited to, employing regression analysis, the Levenberg-Marquardt algorithm, gradient descent, or the Newton-Raphson method to minimize error, especially in applications where parameter relationships are more complex and nonlinear.

[0059] It is desirable to perform the same function of determining material properties of a target described above using high-resolution spectral measurements at a large number of wavelengths, but delivering this same capability using limited illumination and image sensing devices, such as those found in common personal mobile devices, is difficult. A common multi-channel image sensor (e.g., a Red-Green-Blue (RGB) image sensor) provides only three values to work with, making it mathematically infeasible to extract a full, detailed spectral reflectance from only the red, green, and blue values output by an RGB image sensor. By modeling the target being imaged, the dimensionality of the problem is reduced and determination of target properties is possible, when properly segregated and mathematically mapped.Overview of Fingerprint Generation

[0060] Figure 5 illustrates a process 500 for differentially illuminating an object for generation of a reflectance spectrum of the object. An illuminant - or multiple illuminants - can initially strobe through multiple color channels. Assume, for example, that the display of a computing device will serve as the source of illumination as shown in Figure 7. In such an embodiment, the display can perform a series of discrete illumination events by illuminating the object with a first colored light (e.g., red) by activating at least some of its illuminants (e.g., LEDs) (step 501 ), illuminating the object with a second colored light (e.g., green) by activating at least some of its illuminants with (step 502),and illuminating the object with a third colored light (e.g., blue) by activating at least some of its illuminants (step 503). In some embodiments, the illuminants are activated with varying intensities.

[0061] In some embodiments, the entire display is illuminated for each illumination event, but the intensity of each illumination event performed for a given colored light might be different. For example, if the display includes an array of LEDs, those LEDs could emit red light at 90 percent, 80 percent, 70 percent, 60 percent, 50 percent, and 40 percent intensity. Similar intensities could be used for green light and blue light, so that the same intensities are used across the different color channels. Those skilled in the art will recognize that a greater or lesser number of illumination events could be performed for each color channel and that the difference in intensity between consecutive illumination events could be greater than ten or less than ten. Generally, the intensity does not fall below a threshold (e.g., 30 or 40 percent) to ensure that the display remains primarily responsible for illuminating the object. In other embodiments, a percentage of the LEDs in the display is used as a proxy for intensity. For example, rather than adjust the intensity of the individual LEDs, the total number of LEDs being illuminated could serve as a reference for intensity (e.g., for 80 percent intensity, 80 percent of the LEDs could be illuminated).

[0062] Each illumination event can vary in length (also called its “duration”).Illumination events produced in the context of photography tend to range from 15 - 100 ms. However, some illumination events could have sub-millisecond durations, while other illumination events could have multi-second durations.

[0063] A multi-channel image sensor may capture a series of digital images in conjunction with the series of illumination events (step 504). Specifically, the image sensor may capture at least one digital image under the colored light produced for each illumination event. Following step 504, several subsets of digital images will be available to the characterization module, namely, a first subset of digital images that are associated with the first color channel, a second subset of digital images that are associated with the second color channel, and a third subset of digital images that are associated with the third color channel. Each digital image in the first subset may correspond to the first color channel at a different intensity. Similarly, each digital imagein the second and third subsets may correspond to the second and third color channels at a different intensity, respectively. A characterization module can then generate a reflectance spectrum for the target based on the spectral characteristics (step 505) which is described in more detail in Figures 6 and 10. The module may analyze the series of digital images in order to produce a reflectance spectrum.

[0064] Figure 6 illustrates a process for characterizing a material property of a target. Note that, for the purpose of illustration, embodiments may be described in the context of skin color determination and skin chromophore detection. However, aspects of those embodiments are generally extendable to other applications. An object is differentially illuminated by an illuminant (step 601 ) as described in more detail in Figure 5. In some embodiments, the object is skin. In some embodiments, the differential illumination comprises different intensities of the illuminant. In some embodiments, the differential illumination comprises different colors of the illuminant. An image sensor generates digital images in conjunction with differential illumination of an object. A characterization module acquires the digital images (step 602). With the combination of spectral output of the known illuminant and the spectral response (color values) of the digital images of the image sensor, the characterization module may generate a reflectance spectrum or a “fingerprint” (step 603) which is described in more detail in Figure 10. In some embodiments the color values of the digital images are red, green, and blue.

[0065] In some embodiments, a value for a parameter is established that lessens or minimizes a difference between the reflectance spectrum and a reference spectrum that is associated with the object (step 604). Lessening the difference may comprise determining a value for the parameter which is less than a threshold value. In some embodiments, establishing may comprise adjusting the value of the parameter across a range, between a lower threshold and an upper threshold, to create modified reflectance spectrums. Each modified reflectance spectrum may be compared against the reference reflectance spectrum to produce a metric that is representative of the difference between that modified reflectance spectrum and the reference reflectance spectrum. Adjusting may be performed across an entire range, such as from [0,1] at a fixed interval of thousandths or hundredths. Adjusting may be performed across a portion of a range, beginning at either a lower threshold (e.g., 0) or the upper threshold(e.g., 1, 10, or 100) and continuing, at a fixed interval, until the value of the difference begins to increase after a lowest difference is discovered. When beginning at the lower threshold, an adjustment may increment upwards until determining a minimum difference and then start to increase again. When beginning at an upper threshold, an adjustment may decrement downwards until determining a minimum difference and then start to decrease again.

[0066] The parameter may be one of multiple parameters (e.g., at least two, three, or four parameters) for which values are established to minimize the difference between the reflectance spectrum and the reference reflectance spectrum. The reference spectrum may be a spectral model of an object similar to that described in Figure 2.

[0067] After step 604, a property of the object may be characterized based on the value established for the parameter (step 604) in some embodiments. In some embodiments the value of the parameter where the object is skin may reflect the property of blood volume, blood oxygenation, water content, melanin content, or bilirubin content. The parameter values may be representative of the properties of the material.

[0068] In order to differentially illuminate an object, illumination from an ilium inant that has a known illumination spectrum defined in a suitable color space is needed. Consider, for example, the computing device 700 shown in Figure 7. The computing device 700 includes a display 702 and a front-facing camera 704 that are situated within a housing 706. Here, the computing device 700 is a mobile phone. However, those skilled in the art will recognize that the approaches discussed below could be readily adapted for other types of computing devices, such as wearable electronic devices (e.g., watches or fitness trackers), tablet computers, and laptop computers. As shown in Figure 7, the front-facing camera 704 is usually set within an aperture or notch in the display 702 but could be offset from the display 702 entirely. The front-facing camera 704 is typically one of multiple cameras included in the computing device 700. For example, the computing device 700 may also include one or more rear-facing cameras 708 and an illuminant 710 that are also situated within the housing 706. The illuminant 710 may be designed to produce a single color - such as a white LED that emits broadspectrum light for the purpose of flooding a scene with a “flash” - or multiple colors, asin systems that use red, green, and blue LEDs in combination to create a wider range of hues through controlled blending.

[0069] The spectral output of the known illuminant may consist of output from the red channel (Ir), from the green channel (Ig), from the blue channel (Ib), and from the white channel (Iw). The spectral output is described further in Figure 8.

[0070] While the front- and rear-facing cameras 704, 708 generally include the same type of image sensor - for example, a complementary metal-oxide semiconductor (CMOS) image sensor - the image sensors need not be identical to each other, In fact, the image sensors generally have different capabilities. For example, the image sensors in front-facing cameras may be optimized for compact size, low power consumption, and decent low-light performance, while the image sensors in rear-facing cameras tend to be larger, more advanced, and may include technologies light Backside Illumination (BSI), Quad Bayer or Tetracell pixel layouts for binning, or stacked sensor designs, allowing for higher resolution and better dynamic range. The image sensor can receive reflected light from the scene being imaged, including a target object or material.

[0071] In some embodiments the image sensor is a Red-Green-Blue (RGB) image sensor. For each of the digital images of the image sensor, the image sensor may output a response from the image sensor’s red channel (Sr), green channel (Sg), and blue channel (Sb). The spectral responses are described further in Figure 9.

[0072] The display 702 and illuminant 710 can serve as known illuminants for the front-facing camera 704 and rear-facing cameras 708, respectively. If the front-facing camera 704 generates a digital image in conjunction with light being emitted by the display 702, the illumination spectrum of the display 702 may be known to, and controllable by, a characterization module that is executing on the computing device 700. Similarly, if one or more of the rear-facing cameras 708 generates a digital image in conjunction with light being emitted by the illuminant 710, the illumination spectrum of the illuminant 710 may be known to, and controllable by, the characterization module.

[0073] Note that, in some embodiments, the known illuminant is a discrete source of light that is separate from the existing illuminants (e.g., the display 702 and illuminant 710) of the computing device 700. Consider, for example, a mobile phone that includes an existing illuminant situated along its rear side to provide illumination in the form of a“flash” while capturing digital images, like the one shown in Figure 7. The known illuminant could be located proximate to this existing ilium inant, or the known illuminant could replace this existing illuminant - in which case the known illuminant may be configurable to produce white light. For example, the known illuminant might be electrically and / or communicatively connected to the computing device 700. In other embodiments, the known illuminant is representative of an existing illuminant. As discussed above, the display 702 of the computing device 700 could be used as the known illuminant, so long as each emitted color is independently controllable. Similarly, the illuminant 710 could be used as the known illuminant, so long as each emitted color is independent controllable.

[0074] A characterization module can acquire a series of digital images that are captured in conjunction with differential illumination from the known illuminant. Said another way, the characterization module can obtain a series of digital images in which amounts of illumination from the known illuminant are varied. In some embodiments the illumination is varied by color. In some embodiments the illumination is varied by intensity. In some embodiments, the characterization module may prompt the generation of the series of digital images and emittance of the different amounts of illumination (e.g., by sending appropriate instructions to the display 702 and front-facing camera 704, or by sending appropriate instructions to the illuminant 710 and rear-facing camera 708). In other embodiments, the characterization module may retrieve the series of digital images from a memory (e.g., a local memory on the computing device 700, or a remote memory that is accessible to the computing device) if, for example, the series of digital images was already generated. Regardless of its approach to acquiring the series of digital images, the characterization module can record a series of digital images in which the illumination is varied.

[0075] Figure 8 illustrates an example set of spectral outputs of a known illuminant. In some embodiments, the known illuminant is either the display 702 or illuminant 710 of the computing device 700 of Figure 7. If, for example, the display 702 serves as the known illuminant, the known illuminant may be a multi-channel illuminant that is able to produce light with distinct characteristic ranges within the visible spectrum. As an example, the display 702 may perform four illumination events by generating, insuccession though not necessarily in this order, red light, green light, blue light, and white light. In such a scenario, the four illuminations comprise red (Ir), green (Ig), blue (Ib), and white (Iw) spectral outputs from the known illuminant. However, a number of illumination outputs can be used depending on the nature of the known illuminant.During measurement, each channel is activated in turn (or in combination), and the reflected light is detected — often through a sensor with its own spectral sensitivity.

[0076] Figure 9 illustrates an example set of spectral responses of an image sensor. In some embodiments, the image sensor is the front-facing camera 704 or rear-facing camera 708 of the computing device 700 of Figure 7. Consider, for example, a scenario where the display 702 serves as the known illuminant while the front-facing camera 704 serves as the known image sensor. In such a scenario, the front-facing camera 704 may be instructed to generate digital images in conjunction with illumination events performed by the display 702. As mentioned above, the image sensor is preferably a multi-channel sensor and the spectral response of each channel is characterized by distinct sensitivity curves across the visible wavelength range, typically spanning from approximately 400 nm to 700 nm. In some embodiments, the channels are red (Sr), green (Sg), and blue (Sb). This design allows the image sensor to record color images by integrating reflected light for each channel across the spectrum, with each channel contributing distinct information to the sensor’s overall RGB output. The specific spectral response profiles of these channels are critical for accurate color measurement and material identification, as they directly influence how the sensor detects and interprets the spectral composition of incoming light under different illumination conditions.

[0077] For the purpose of illustration, a scenario will be further discussed below in which the multi-channel light source generates four spectral outputs - namely, flashes of red light, green light, blue light, and white light - while the multi-channel image sensor generates a digital image in conjunction with each output. If the multi-channel image sensor is an RGB image sensor or three separate photodiodes designed to detect light in the red, green, and blue regions of the electromagnetic spectrum, then there will be 12 total channels of data (i.e., four spectral outputs multiplied by three sensor channels) that can be used to generate an image fingerprint.

[0078] Figure 10 includes a schematic diagram of the approach set forth above for establishing a fingerprint of a target. At a high level, a characterization module can initially record digital images of an object 1008 that are generated by an image sensor 1004 of a computing device 1002 while the object 1008 is differentially illuminated by an illuminant 1006 of the computing device. Each of the digital images can be generated in conjunction with a different color and / or intensity of illumination by the illuminant 1006. For example, the illuminant 1006 may be a single-channel illuminant that is able to emit a single color of light (e.g., white), in which case each of the digital images may be associated with a different intensity of illumination. As another example, the illuminant 1006 could be a multi-channel illuminant - or the illuminant 1006 could be one of multiple single-channel illuminants - that is able multiple colors of light (e.g., one or more LEDs for red light, one or more LEDs for green light, one or more LEDs for blue light), in which case the digital images could include (i) a first subset of digital images that are captured in conjunction with different intensities of a first colored light (e.g., red), (ii) a second subset of digital images that are captured in conjunction with different intensities of a second colored light (e.g., green), (iii) a third subset of digital images that are captured in conjunction with different intensities of a third colored light (e.g., blue), etc.

[0079] Thereafter, the characterization module can estimate, for the object 1008, a reflectance spectrum based on an analysis of the red, green, and blue values of the digital images as described in Figure 8 and a known illumination spectrum as described in Figure 9 that is associated with the illuminant 1006. For example, the characterization module may establish, based on the red, green, and blue values of the digital images, appropriate red, green, and blue values for the object on a per-pixel basis.

[0080] The process for generating sensor output values in a spectral imaging system involves calculating the sum of element-wise products of three key components: the skin's reflectance spectrum, the illumination spectrum, and the sensor's spectral response. Where the reflecting surface being measured is Rs (“reflectance of skin”), the image sensor output Fn(where n is r, g, or b) is the sum of elements of the illuminant spectral output vector and the image sensor spectral response. Here eachmultiplication, denoted with *, is the “element-wise” multiplication, also known as the “hadamard product,”Fn = sum( Rs * Im * Sn) Eq. 2where “m” is one of the illuminations of the illuminant 1006 (e.g., Ir for red, Igfor green, Ibfor blue, Iwfor white) and “n” is the output of one of the image sensors (e.g., Sr for red, Sb for blue, Sgfor green) when illuminating the skin with a given illuminant. For example, the below equation represents the blue channel digital image output when illuminating the skin Rs with the green illuminant.Fgb = sum( Rs * lg* Sb) Eq. 3

[0081] A four-channel light source and three-channel image sensor yields twelve total output values that are calculated according to Equation 2. In other embodiments, a fingerprint may have another set of values depending on the number of illuminant channels and number of sensor channels.

[0082] In embodiments where the display of the computing device 1002 is used as the illuminant 1006 and the image sensor 1004 is included in a front-facing camera of the computing device 1002, information to facilitate the aforementioned approach could be presented either visually (e.g., via the display) or audibly (e.g., via an audio output mechanism). For example, the characterization module may indicate, through an interface, that the computing device should be positioned and oriented such that a given object (e.g., a face) is viewable by the front-facing camera. Moreover, the characterization module may indicate, through the interface, whether the computing device needs to be positioned nearer to, or further from, the given object. As discussed above, the image sensor 1006 should ideally be within a certain proximity of the given object to ensure that changes in color and / or intensity of light emitted by the illuminant 1006 are suitably observable by the image sensor 1004.

[0083] An important advantage of the reflectivity modeling method is the ability to produce a large number of near real-world reflectance spectra and use them to optimize transformations from the multiple-illuminated image sensor values into the modeled target properties. In order to use only the spectral fingerprint to determine material properties, the process discussed below with respect to Figure 11 can be utilized.Overview of Target Property Determination

[0084] Figure 11 describes a process 1100 to determine a target property based on an analysis of values output by a multi-channel image sensor. A plurality of reflectance spectrums is generated by permuting one or more parameters of a spectral model, the shape of which provides insights into one or more materials that constitute an object (step 1101 ). A number of parameters are chosen. In some embodiments, a number parameters of the spectral model are held constant. In some embodiments, a large number of parameters are chosen to produce a plurality of training reflectances, where the plurality of training reflectances may be more than a hundred or a thousand.

[0085] For each of these reflectance spectrums, a spectral fingerprint is calculated, resulting in a plurality of spectral fingerprints (step 1102). The calculation of an individual fingerprint is described in more detail in Figure 10. Based on this plurality of spectral fingerprints, a transform for each parameter of the spectral model is optimized independently (step 1103).

[0086] In some embodiments, the transform may be performed with a least-squares optimized linear transform (matrix) method. To do so, an over-defined system of equations is created where linear combinations of the values in each fingerprint are set equal to the spectral model parameter being solved for. Each fingerprint F may have the number of values equal to the number of illuminants times the number of sensor channel values. Accordingly, if an RGB image sensor is instructed to capture digital images in conjunction with red, green, blue, and white illumination events, then the fingerprint will include 12 entries (i.e., three sensor channels multiplied by four illumination channels). Each value is associated with a reflectance r, illuminant channel i, and sensor channel s. The reflectance r was generated with model parameter P and is the n-th member of the training reflectance set.Xl*Fris + X2*Fris + X2*Fris... + Xn*Fris = Pn Eq. 4

[0087] With an over-defined list of equations derived from the plurality of spectral fingerprints, an ordinary least squares multiple linear regression is applied to the system to determine the optimum coefficients Xi which correspond to the number of fingerprintvalues. The vector [xi, x2, x3, x4, x5, x6, x7, x8, x9, xio, xn, xi2] can then be used to determine the material property P based only on the spectral fingerprint F using Equation 5.Pf = Xl*Fl + X2*F2+ X3*F3 + X4*F4+ X5*F5 + X6*Fe + X7*F7+ X8*Fs + X9*Fg + X1O*FIO + Xl1*Fl1 + X12*F12 Eq. 5

[0088] In some embodiments, the transform for each parameter to be optimized may be performed using other methods. These methods may include, but are not limited to, neural networks, non-linear searches, Monte Carl (random sampling), etc. Each optimized transform is then stored in memory (step 1104).

[0089] Figure 12 illustrates a set of training reflectance spectrums that could be utilized in the process 1100 of Figure 11. For this example, blood oxygenation, blood volume, and melanin were selected as the input parameters for the spectral model of skin. Other parameters of the spectral model were held constant. For each input parameter, 3 values were chosen, yielding 27 training reflectances in total as shown in Figure 12. Those skilled in the art will recognize that a relatively small number of parameters and values were selected for the purpose of illustration. In practice, tens, hundreds, or even thousands of training reflectance spectrums could be used (e.g., increasing to the point of diminishing return). The complete equation set for the 27 training reflectance spectrums when solving for one distinct parameter value P is a system of equations based on Equation 4. The complete set of equations for the 27 training reflectance spectrums is set forth below.Xl*Fl,w,r+X2*Fl,w,g+X3*Fl,wb+X4*Fl,r,r+X5*Fl,r,g+X6*Fl,r,b+X7*Fl g,r+X8*Fl,g,g+X9*Fl,g,b+Xl0*Fl,b,r+Xl1*Fl,b,g+Xl2*Fl,b,b = Pl Xl*F2lw,r+X2*F2lw,g+X3*F2,wb+X4*F2,r,r+X5*F2lr,g+X6*F2lr,b+X7*F2g,r+X8*F2,g,g+X9*F2,g,b+Xl0*F2,b,r+Xl1*F2,b,g+Xl2*F2lb,b = P2 Xl*F3,w,r+X2*F3,w,g+X3*F3,w,b+X4*F3,r,r+X5*F3,r,g+X6*F3,r,b+X7*F3,g,r+X8*F3,g,g+X9*F3,g,b+Xl0*F3,b,r+Xl1*F3,b,g+Xl2*F3,b,b = P3 Xl*F41w,r+X2*F41w,g+X3*F4,wb+X4*F4,r,r+X5*F4,r,g+X6*F41r,b+X7*F4g,r+X8*F4,g,g+X9*F4,g,b+Xl0*F4,b,r+Xl 1*F4lb,g+Xl2*F4lblb = P4Xl*F5,w,r+X2*F5,w,g+X3*F5,wb+X4*F5,rlr+X5*F5,r,g+X6*F5,r,b+X7*F5g,r+X8*F5,g,g+X9*F5,g,b+Xl0*F5,b,r+Xl1*F5,b,g+Xl2*F5,b,b = P5 Xl*F6,w,r+X2*F6,w,g+X3*F6,wb+X4*F6,r,r+X5*F6,r,g+X6*F6,r,b+X7*F6g,r+X8*F6,g,g+X9*F6,g,b+Xl0*F6,b,r+Xl1*F6,b,g+Xl2*F6,b,b = Pe Xl*F7lw,r+X2*F7lw,g+X3*F7,wb+X4*F7,r,r+X5*F7,r,g+X6*F7lr,b+X7*F7g,r+X8*F7,g,g+X9*F7,g,b+Xl0*F7,b,r+Xl1*F7,b,g+Xl2*F7lb,b = P7 Xl*F8,w,r+X2*F8,w,g+X3*F8,w,b+X4*F8,r,r+X5*F8,r,g+X6*F8,r,b+X7*F8,g,r+X8*F8,g,g+X9*F8,g,b+Xl0*F8,b,r+Xl1*F8,b,g+Xl2*F8,b,b = Ps Xl*F9]w,r+X2*F9]w,g+X3*F9,wb+X4*F9,r,r+X5*F9]r1g+X6*F9]r,b+X7*F9g,r+X8*F9,g,g+X9*F9,g,b+XlO*F9,b,r+Xn*F91b,g+Xl2*F9]b1b=PgAs mentioned above, the vector [xi, x2, x3, x4, x5, x6, x7, x8, x9, xio, xn, X12] can then be used to determine the parameter P based only on the spectral fingerprint F using Equation 5.Overview of Temporal Tracking Applications

[0090] Figure 13 describes a process 1300 to determine a change in a property of an object over time. Note that, for the purpose of illustration, embodiments may be described in the context of skin color determination and skin chromophore detection. However, aspects of those embodiments are generally extendable to other applications.

[0091] Initially, a characterization module acquires a first plurality of digital images of an object (step 1301). The first plurality of images are generated by an image sensor with a plurality of sensor channels while the object is differentially illuminated by a light source with a plurality of color channels. Each of the first plurality of digital images is generated in conjunction with illumination by a different one of the plurality of color channels. The differential illumination and image generation processes are as described previously. The characterization module generates a first spectral fingerprint that corresponds to a first time at which the first plurality of digital images are captured for the object (step 1302). The first spectral fingerprint can be generated by element-wise multiplying spectral responses of the plurality of sensor channels across the first plurality of digital images with illumination spectra of the plurality of color channels. The first fingerprint generation process may be as described previously. The first spectral fingerprint can then be stored in a memory (step 1303) Note that the memory could be internal to the computing device that generates the images or external to the computing device that generates the images (e.g., a network-accessible storage medium).

[0092] Thereafter, the characterization module can acquire a second plurality of digital images of an object (step 1304). The second plurality of images are generated by an image sensor with a plurality of sensor channels while the object is differentially illuminated by a light source. In some embodiments, a notification is generated to prompt capture of the second plurality of digital images in response to a determination that a predetermined amount of time has elapsed since the first time. The notification may be generated by the characterization module. The notification may be visual, audible, or another sort of prompt on a computing device.

[0093] The characterization module generates a second spectral fingerprint that corresponds to a second time at which the second plurality of digital images are captured for the object (step 1305). The second spectral fingerprint can be generated by element-wise multiplying spectral responses of the plurality of sensor channels across the second plurality of digital images with the illumination spectra of the plurality of color channels. The first fingerprint generation may be as described previously.

[0094] The characterization module can then establish whether there has been a change in a property of the object based on a comparison of the second spectralfingerprint to the first spectral fingerprint (step 1306). In some embodiments, an indication of the change in the property of the object is displayed on the display of a computing device. In some embodiments, an indication of the change in the property of the object is transmitted to a destination external to the computing device. In some embodiments, a change may be the size, shape, color, or texture of a mole. In this case, the change in the property may be automatically conveyed to a healthcare system, assuming the user has granted the appropriate permissions, in order to facilitate diagnosing or scheduling of an appointment. In other embodiments, a change in condition of an object is conveyed to the manufacturer to notify them of the change in condition or quality of the material. Accordingly, the characterization module may be responsible for transmitting, to a destination external to the computing device that generates the digital images, an indication of the change in the property of the object.

[0095] Note that the first and second pluralities of digital images are usually generated by the same computing device, for example, an individual’s mobile phone. While the first and second pluralities of digital images could be generated by different computing devices, this can make the comparison less reliant as those computing devices may be associated with different illumination spectra and different sensor spectra.

[0096] Figure 14 describes a process, similar to that of process 1300 of Figure 13, for determining a change in a skin property. Note that, for the purpose of illustration, embodiments may be described in the context of skin chromophore detection, specifically that of bilirubin. However, aspects of those embodiments are generally extendable to other applications. An application is provided for execution on a computing device comprising a processor, an illumination source, and at least one image sensor. An application of a computing device, such as a phone, tablet, or other device, is configured to capture a baseline fingerprint of an individual’s skin (step 1401). The baseline measurement may be used to determine a range of skin properties (step 1402b). After waiting for a predetermined period of time (step 1402a), a second spectral fingerprint is captured. In some embodiments, the period of time may be such that it allows for a reading before and after a feeding of an infant. The first fingerprint and the second fingerprint are compared to determine the change in a parameter representing aproperty of the skin (e.g. bilirubin). The application may be configured to display the resulting change in a skin property (step 1404), like bilirubin levels of an infant.

[0097] In some embodiments, the process 1400 is used to determine the material property, specifically bilirubin, from a sensor value. A collection of training model reflectance spectra are produced by varying a single model parameter, such as bilirubin. All other parameters are held constant. For each training reflectance, a corresponding spectral fingerprint is computed and used to determine the material property based on sensor input as described previously. Process 1400 utilizes the transform to map a first and second fingerprint with a relative change to a material property, such as bilirubin. Accordingly, by comparing a first and second spectral fingerprint measured at different time points, the process quantifies the relative change in the target parameter. A similar procedure can be utilized for isolation of melanin or any other of the other chromophores in the model. Thus, the aforementioned method is not limited to melanin or bilirubin. This provides unique image fingerprints for each user, which can be tracked over time to monitor changes.

[0098] The ABCDE guide has been commonly used as an early screening tool for detecting skin cancers, with a detection capability exceeding the 90% and false positives in the 30% range and false negatives in the 10-20%. Hence it is recognized as an effective first level self-screening method. ABCDE stands for “Asymmetry,” “Border,” “Color,” “Diameter," and "Evolving.” “A” is characterized by one half of the mole not matching the other. Benign moles are usually symmetrical, while malignant moles - like those that accompany melanoma - are often irregular. “B” is characterized by irregular, uneven, or jagged edges. Benign moles tend to have smooth, well-defined borders, while malignant moles more often have blurred or scalloped edges. “C” is characterized by the presence of multiple colors within the same lesion (brown, black, red, white, blue). Benign moles are typically one uniform color, whereas malignant moles often show color variation. “D” is characterized by the size of the mole, specifically if the diameter is larger than 6 mm. While smaller lesions can still be malignant, a larger size is generally indicative of increased risk. “E” is characterized by changes in size, shape, color, or texture overtime. New symptoms like itching, bleeding, or crusting mayappear. Out of these, “E” is recognized as the most significant signal, and combining this with any of the others should signal immediate dermatologist consultation.

[0099] Figure 15 describes a process for carrying out ABDCE evaluation for skin melanoma detection. Temporal monitoring of ABODE overtime can be improved by using a quantifiable technique. In order to accurately evaluate ABDCE over time, both an accurate geometrical measurement and color measurement are necessary. Once a suspicious region of the skin has been characterized, then daily scanning can be used to determine whether to seek immediate medical advice.[000100] In some embodiments, a computing device includes instructions which guide a user through the ABDCE evaluation process described in Figure 15. The evaluation is initiated to assess a suspicious region which may contain a mole (step 1501). First, it is determined whether the mole changed in size, shape, color, or symptoms (itching, bleeding, crusting) (step 1502). If yes, the user is advised to seek medical evaluation (step 1502a). If no, the evaluation proceeds to the next step. It is determined whether one half of the mole looks different from the other (step 1503). It is determined whether the edges are irregular, blurry, or jagged (step 1504). It is determined whether the mole has multiple colors (brown, black, red, white, blue) (step 1505). The size of the mole is determined and it is established whether it is larger than 6mm (step 1506). If the mole is larger than 6mm, the user is advised to seek medical evaluation (step 1506a). If not, the evaluation continues to the decision outcomes (1507).[000101] The decision outcomes may be as follows:• If “E” is present, the user is advised to seek medical evaluation.• If 2 or more factors (A, B, C, D) are present, it is highly suspicious and the user is advised to see a dermatologist.• If only 1 minor change is noted, the user is advised to monitor the suspicious region closely and document further changes.• If no ABODE criteria apply, it is likely benign, but the user is advised to continue with daily evaluation.[000102] The precision of skin lesion assessment depends significantly on the analytical tool employed. For shape analysis (asymmetry, border, diameter), visual inspection offers moderate precision (50-100 pm), dermoscopy achieves high precision(10-30 m), and biopsy provides near-perfect precision (1-5 pm). Color analysis varies from coarse resolution with the human eye (AE ~ 2-3), to fine detail with dermoscopy or Al-based assessment (AE < 1), and reaches ultra-precise differentiation with histopathology (AE ~ 0.5). When monitoring changes or evolution over time, imaging tools are essential for reliably detecting subtle alterations in lesion characteristics.[000103] In some embodiments, an application is provided for execution on a computing device comprising a processor, an illumination source, and at least one image sensor. The application is configured to acquire spectral and colorimetric images of a target skin lesion through using the process described in Figure 6, and to analyze said images in accordance with the ABCDE evaluation process 1500. The system utilizes image processing algorithms to quantify geometric and chromophore-related features, thereby assessing each ABCDE parameter on the sampled lesion. The application further enables temporal tracking by storing historical spectral measurements, including full or partial reflectance curves and chromophore estimations, and algorithmically comparing new measurements to previous data sets. The process incorporates automated or user-enabled reminders to re-acquire images at defined intervals, supporting longitudinal analysis of changes in lesion morphology or spectral properties. The results of both the ABCDE evaluation and temporal spectral comparisons are displayed via the user interface, optionally alerting the user or healthcare provider to clinically significant changes or trends that may warrant further evaluation. Combining advanced photogrammetry and colorimetry methods, particularly when supported by Al and time-based tracking, enables a level of diagnostic precision that surpasses the capabilities of individual tools.[000104] In further embodiments, the approach discussed above may support skin color assessment and skin-condition trending over time. The characterization module could compute device-independent color metrics (e.g., CIE L*a*b*, A£2ooo) for regions of skin selected via gaze or by predefined anatomical templates. Using the ambient-subtracted, actively illuminated measurements described above, the characterization module can estimate biophysical indices such as relative melanin and hemoglobin content, erythema, pallor, jaundice proxies, bruise evolution, inflammation, dryness, or post-procedure changes. Measurements can be time-stamped and / or registered to apose or geography, so as to enable longitudinal comparisons at the same anatomical region of interest (ROI), with optional automatic re-acquisition using visual or fiducial alignment cues. Various types of interfaces could be used to convey this information. As an example, an augmented reality (AR) overlay of instantaneous color (e.g., a small swatch and numeric L*a*b*) could be presented together with trend indicators (e.g., AE to baseline, weekly slope, confidence bounds). For clinical or cosmetic workflows, the characterization module could enforce standardized capture protocols (e.g., stand-off distance, ROI size, exposure bounds), and maintain on-device baselines so that progress can be evaluated independently of changing ambient light.[000105] To further improve measurement fidelity, the multi-channel light source may implement temporal modulation of each discrete wavelength and the multi-channel image sensor pipeline may implement synchronous demodulation (lock-in detection). For example, each illuminant (e.g., LED, VCSEL, or laser) can be driven with a distinct square-wave or sinusoidal carrier (e.g., fi, f2,..., fk) or with orthogonal code sequences (e.g., Walsh / Hadamard, m-sequences), while the image sensor readout performs matched filtering to recover per-wavelength responses. This modulation framework suppresses unmodulated ambient light and mitigates flicker from mains lighting. It also allows simultaneous multi-line emission without inter-channel crosstalk. In rolling-shutter cameras, the characterization module - acting as a controller - aligns exposure windows to illumination plateaus or employs phase dithering across frames to average out line-scanning artifacts. In some embodiments, chirped or spread-spectrum drives are used to distribute optical power and avoid perceptible flicker while maintaining narrowband recoverability.[000106] In addition, polarization control could be incorporated in the illumination path and / or and sensor path to increase signal-to-noise ratio (i.e., reduce unwanted specular and subsurface components). A linear polarizer could be placed in front of the light source, and an analyzer (e.g., a crossed or variable liquid-crystal polarization rotator) could be placed in front of the image sensor. Cross-polarized capture preferentially rejects specular (surface) reflections from skin oils, isolating diffuse / subsurface reflectance that encodes melanin and hemoglobin signatures; parallel-polarized capture can be used when surface sheen is diagnostically relevant(e.g., dryness, scaling). By acquiring a short sequence of multi-polarization frames (e.g., 0°, 45°, 90°, 135°) under known wavelengths, the characterization module can estimate Stokes parameters or derive a specular / diffuse separation, improving the stability of color estimates and biophysical indices across variable skin types and viewing angles. Polarization state may be time-multiplexed in coordination with wavelength modulation so that demodulation yields per-wavelength, per-polarization signals for robust inversion.[000107] For longitudinal skin applications - namely, those where the same skin area is imaged and characterized repeatedly (e.g., over hours, days, weeks, months, or years) to track changes - the characterization module may fit a simple layered-skin model (epidermal melanin + dermal hemoglobin with scattering) to the ambient-corrected, polarization-aware measurements, yielding device-independent parameters that are tracked over time. The characterization module may flag statistically significant deltas (e.g., 42oooorparameter change exceeding a configured threshold) and display them in AR at the ROI along with recommended standardized re-measurement prompts. Said another way, the characterization module may visually identify region(s) in which the property of a target (e.g., skin) changed by at least a given amount through modification of an AR overly, which may be generated by the same computing device used to generate digital images of the target or another computing device. Privacypreserving modes may allow all trend data to be kept on the computing device used for imaging, unless explicitly exported. Together, the addition of temporal modulation and polarization control enhances SNR and measurement repeatability, enabling reliable skin color and skin-condition change detection as part of the broader AR-based colorimetry and spectroscopy platform. Potential applications include, but are not limited to, (i) following how a bruise, rash, or post-procedure redness fades; (ii) monitoring hyperpigmentation lightening under treatment; (iii) watching acne inflammation or dryness improve or worsen through a regimen; (iv) tracking a region of skin change for early signs of cancer using the ABCDE evaluation; and (v) tracking £2000fromabaseline at the exact same ROI to quantify change.Overview of Characterization Module[000108] Figure 16 illustrates a network environment 1600 that includes a characterization module 1604 executing on a computing device 1602. The computing device 1602 could be, for example, computing device 700 of Figure? or computing device 1000 of Figure 10. In some embodiments, the characterization module 1604 is an integral part of the computing device 1602 on which it resides. For example, the characterization module 1604 may be part of, or accessible to, the operating system of the computing device. In other embodiments, the characterization module 1604 is representative of a computer program that can be selectively executed by the computing device, for example, upon receiving input indicative of a request to determine the color and shade of a target object of interest.[000109] Individuals may be able to interface with the characterization module 1604 via an interface 1606. As discussed above, the characterization module 1604 may be responsible for driving illuminants to illuminate a target object and then examining digital images of the target object generated by an image sensor to establish characteristics thereof. As discussed above, the illuminants and image sensor could be part of the computing device 1602. The characterization module 1604 may also be responsible for creating or supporting interfaces through which an individual can view the digital images, initiate post-processing operations, manage preferences, etc.[000110] The characterization module 1604 may reside in a network environment 1600 as shown in Figure 16. Thus, the computing device 1602 that executes the characterization module 1604 may be connected to one or more networks 1608a-b. The networks 1608a-b can include personal area networks (PANs), local area networks (LANs), wide area networks (WANs), metropolitan area networks (MANs), cellular networks, the Internet, etc.[000111] Generally, the characterization module 1604 resides on the same computing device 1602 as the image sensor and known illuminant. For example, the characterization module 1604 may be part of a mobile application through which an image sensor of a mobile phone can be operated. In other embodiments, the characterization module 1604 is communicatively coupled to the image sensor and / or the known illuminant across a network. For example, the characterization module 1604 may be executed by a platform (also referred to as a “cloud platform”) that resides on acomputer server and that is accessible via a network (e.g., the Internet), and the platform may acquire digital images from another computing device (e.g., a mobile phone) via the network.[000112] In some embodiments, the characterization module 1604 is executed by a cloud computing service operated by Amazon Web Services®, Google Cloud Platform™, Microsoft Azure®, or a similar technology. In such embodiments, the characterization module 1604 may reside on a computer server that is communicatively coupled to one or more other computer servers 1610. The other computer servers 1610 can include color mixing models, items necessary for post-processing such as heuristics and algorithms, and other assets.Overview of Processing System[000113] Figure 17 is a block diagram illustrating an example of a processing system 1700 that is capable of implementing the operations described herein. For example, components of the processing system 1700 may be hosted on a computing device such as a smart phone, tablet, computer or other computing device described in Figures 7 and 10. The computing device may include the known ilium inant, image sensor, and a characterization module, where the characterization is described in Figure 16.[000114] The processing system 1700 may include a processing unit 1702, main memory 1706, non-volatile memory 1710, network adapter 1712, display 1718, input / output device 1720, control device 1722 (e.g., a keyboard or pointing device), drive unit 1724 including a storage medium 1726, and signal generation device 1730 that are communicatively connected to a bus 1716. The bus 1716 is illustrated as an abstraction that represents one or more physical buses or point-to-point connections that are connected by appropriate bridges, adapters, or controllers. The bus 1716, therefore, can be a system bus, a Peripheral Component Interconnect (PCI) bus or PCI-Express bus, a HyperTransport or industry standard architecture (ISA) bus, a small computer system interface (SCSI) bus, a universal serial bus (USB), inter-integrated circuit (l2C) bus, or an Institute of Electrical and Electronics Engineers (IEEE) standard 1394 bus (also referred to as “Firewire”).[000115] The processing unit 1702 can have generic characteristics similar to general-purpose central processing units (CPUs) or graphical processing units (GPUs), or the processor may be an application-specific integrated circuit (ASIC) that provides control functions to the computing device of which the processing system 1700 is a part.[000116] While the main memory 1706, non-volatile memory 1710, and storage medium 1726 are shown to be a single medium, the terms “machine-readable medium” and “storage medium” should be taken to include a single medium or multiple media (e.g., a centralized / distributed database and / or associated caches and servers) that store one or more sets of instructions 1704, 1708, 1728. The terms “machine-readable medium” and “storage medium” shall also be taken to include any medium that is capable of storing, encoding, or carrying instructions for execution by the processing system 1700.[000117] In general, the routines executed to implement the embodiments of the disclosure may be implemented as part of an operating system or a computer program. Computer programs typically comprise one or more instructions (e.g., instructions 1704, 1708, 1728) set at various times in various memory and storage devices in a computing device. When read and executed by the processing unit 1702, the instructions cause the processing system 1700 to perform operations to execute elements involving the various aspects of the present disclosure.[000118] Further examples of machine- and computer-readable media include recordable-type media, such as volatile memory and non-volatile memory 1710, removable disks, hard disk drives, and optical disks (e.g., Compact Disk Read-Only Memory (CD-ROMS) and Digital Versatile Disks (DVDs)), and transmission-type media, such as digital and analog communication links.[000119] The network adapter 1712 enables the processing system 1700 to mediate data in a network 1714 with an entity that is external to the processing system 1700 through any communication protocol supported by the processing system 1700 and the external entity. The network adapter 1712 can include a network adaptor card, a wireless network interface card, a router, an access point, a wireless router, a switch, a multilayer switch, a protocol converter, a gateway, a bridge, bridge router, a hub, a digital media receiver, a repeater, or any combination thereof.

Claims

CLAIMSWhat is claimed is:

1. A non-transitory medium with instructions stored thereon that, when executed by a processing unit of a computing device that also includes an illuminant and an image sensor, cause the processing unit to perform operations comprising:acquiring digital images of an object that are generated by the image sensor while the object is differentially illuminated by the illuminant, wherein each of the digital images is generated in conjunction with a different color or intensity of illumination by the illuminant; generating, for the object, a reflectance spectrum based on an analysis of red, green, and blue values of the digital images and a known illumination spectrum that is associated with the illuminant;establishing a value for a parameter that minimizes a difference between the reflectance spectrum and a reference reflectance spectrum that is associated with the object; andcharacterizing a property of the object based on the value established for the parameter.

2. The non-transitory medium of claim 1, wherein the object is skin, and wherein the parameter is representative of blood volume, blood oxygenation, water content, melanin content, or bilirubin content.

3. The non-transitory medium of claim 1, wherein the parameter is one of multiple parameters for which values are established to minimize the difference between the reflectance spectrum and the reference reflectance spectrum.

4. The non-transitory medium of claim 1, wherein said establishing comprises: adjusting the value of the parameter across a range, between a lower threshold and an upper threshold, to create modified reflectance spectrums, each of which is compared against the reference reflectance spectrum to producea metric that is representative of the difference between that modified reflectance spectrum and the reference reflectance spectrum.

5. The non-transitory medium of claim 4, wherein said adjusting is performed across the range in its entirety at a fixed interval.

6. The non-transitory medium of claim 4, wherein said adjusting is performed across a portion of the range, beginning at either the lower threshold or the upper threshold and continuing, at a fixed interval, until the value of the difference begins to increase after a lowest difference is discovered.

7. The non-transitory medium of claim 1,wherein the illuminant is a multi-channel illuminant that is able to emit red light, green light, blue light, and white light, andwherein the image sensor is a Red-Green-Blue (RGB) image sensor that outputs, for each of the digital images, (i) a red spectral response, (ii) a green spectral response, and (iii) a blue spectral response.

8. The non-transitory medium of claim 7, wherein the operations further comprise:obtaining, through an analysis of the digital images,(i) a set of four red spectral responses, which correspond to a first digital image generated in conjunction with emittance of red light by the illuminant, a second digital image generated in conjunction with emittance of green light by the illuminant, a third digital image generated in conjunction with emittance of blue light by the illuminant, and a fourth digital image generated in conjunction with emittance of white light by the illuminant,(ii) a set of four green spectral responses, which correspond to the first, second, third, and fourth digital images, and(iii) a set of four blue spectral responses, which correspond to the first, second, third, and fourth digital images.

9. A method performed by a computer program executing on a computing device, the method comprising:acquiring a first plurality of digital images of an object that are generated by an image sensor with a plurality of sensor channels while the object is differentially illuminated by a light source with a plurality of color channels, wherein each of the first plurality of digital images is generated in conjunction with illumination by a different one of the plurality of color channels;generating, for the object, a first spectral fingerprint that corresponds to a first time at which the first plurality of digital images are captured, wherein the first spectral fingerprint is generated by element-wise multiplying spectral responses of the plurality of sensor channels across the first plurality of digital images with illumination spectra of the plurality of color channels;storing the first spectral fingerprint generated for the object in a memory; acquiring a second plurality of digital images of the object that are generated by the image sensor while the object is differentially illuminated by the light source;generating, for the object, a second spectral fingerprint that corresponds to a second time at which the second plurality of digital images are captured, wherein the second spectral fingerprint is generated by element-wise multiplying spectral responses of the plurality of sensor channels across the second plurality of digital images with the illumination spectra of the plurality of color channels; andestablishing whether there has been a change in a property of the object based on a comparison of the second spectral fingerprint to the first spectral fingerprint.

10. The method of claim 9, further comprising:causing display of an indication of the change in the property of the object.

11. The method of claim 10, wherein said causing comprises visually identifying one or more regions in which the property of the object changed by at least a given amount through modification of an augmented reality overlay.

12. The method of claim 11, wherein the augmented reality overlay is generated by the computing device.

13. The method of claim 9, further comprising:generating a notification to prompt capture of the second plurality of digital images in response to a determination that a predetermined amount of time has elapsed since the first time.

14. The method of claim 9, further comprising:transmitting, to a destination external to the computing device, an indication of the change in the property of the object.

15. A method comprising:generating a plurality of reflectance spectrums by permuting one or more parameters of a spectral model, a shape of which provides insights into one or more materials that constitute an object;calculating a spectral fingerprint for each reflectance spectrum of the plurality of reflectance spectrums, so as to calculate a plurality of spectral fingerprints;optimizing, based on the plurality of spectral fingerprints, a transform for each parameter of the one or more parameters of the spectral model independently; andstoring each optimized transform in a memory.

16. The method of claim 15, further comprising:generating the spectral model that is associated with the object by -acquiring multiple spectral responses of a multi-channel image sensor, wherein each of the multiple spectral responses corresponds to a different one of multiple sensor channels of the multi-channel image sensor,acquiring multiple illumination spectra of a multi-channel light source, wherein each of the multiple illumination spectra corresponds to a different one of multiple illumination channels of the multi-channel light source, andgenerating multiple spectral fingerprints by element-wise multiplying the multiple spectral responses with the multiple illumination spectra.

17. The method of claim 16,wherein the multi-channel image sensor is a Red-Green-Blue (RGB) image sensor, andwherein the multiple spectral responses include(i) a red spectral response of a red sensor channel of the RGB image sensor,(ii) a green spectral response of a green sensor channel of the RGB image sensor, and(iii) a blue spectral response of a blue sensor channel of the RGB image sensor.

18. The method of claim 17, wherein the multiple illumination spectra include(i) a red illumination spectrum of a red color channel of the multi-channel light source,(ii) a green illumination spectrum of a green color channel of the multi-channel light source,(iii) a blue illumination spectrum of a blue color channel of the multi-channel light source, and(iv) a white illumination spectrum of a white color channel of the multi-channel light source.

Images