Determining specular reflection information

Abstract

A computer-implemented method 100 is described. The computer-implemented method includes receiving 102 first and second imaging data acquired by an imaging system 204 of a subject 202 illuminated by a first illumination 206a in a first spectral band and a second illumination 206b in a second spectral band with a different spectral content than the first spectral band. The second illumination incident on the subject is polarized. The received first imaging data is acquired in the first spectral band. The received second imaging data is acquired in the second spectral band. Depending on the polarization state of the reflected first and second illumination received by the imaging system after reflection from the subject's surface, the first and second illumination are entered into the imaging system via an imaging system polarizer 310 of the imaging system. Information about the specular reflection from the subject's surface is determined by comparing the first and second imaging data.

Description

[Technical field]

[0001] The present invention relates to a method, a tangible machine-readable medium and an apparatus for use in imaging in certain conditions. [Background technology]

[0002] A topic of interest in the field of non-intrusive measurement and monitoring relates to skin sensing for personal care and health applications. Skin sensing systems have been developed that allow for the quantification of skin and monitoring of skin features that provide users with information that is too small to detect, too subtle to notice, or too slow to follow. To achieve user-acceptable results, such skin sensing systems need to provide sensitivity and specificity in performing skin sensing. Users establish trust in these skin sensing systems when the measurements taken by such skin sensing systems are proven to be robust and reliable. Summary of the Invention [Problem to be solved by the invention]

[0003] Imaging-based skin sensing systems implement various imaging techniques to determine certain information about a user's skin. Such information includes parameters such as the radiance of a user's skin. Certain systems for determining radiance are expensive, complex, and / or bulky. [Means for solving the problem]

[0004] Aspects or embodiments described herein relate to determining certain information, such as the shininess of a subject's surface, and avoid one or more problems associated with the cost, complexity, and / or bulk of systems for determining such information.

[0005] In a first aspect, a method is described. The method is a computer-implemented method. The computer-implemented method includes receiving first and second imaging data acquired by an imaging system of a subject illuminated by a first illumination in a first spectral band and a second illumination in a second spectral band having a different spectral content than the first spectral band. The second illumination incident on the subject is polarized.

[0006] The received first imaging data is acquired within a first spectral band as a result of a first color filter of the imaging system allowing at least a portion of the first spectral band to enter the imaging system and preventing at least a portion of the second spectral band from entering the imaging system, such that a majority of intensity information in the first imaging data acquired within the first spectral band is derived from the first illumination.

[0007] The received second imaging data is acquired within a second spectral band as a result of a second color filter of the imaging system allowing at least a portion of the second spectral band to enter the imaging system and preventing at least a portion of the first spectral band from entering the imaging system, such that a majority of the intensity information in the second imaging data acquired within the second spectral band is derived from the second illumination.

[0008] Depending on the polarization states of the reflected first and second illumination received by the imaging system after reflection from the subject's surface, the first and second illumination are entered into the imaging system via an imaging system polarizer of the imaging system, such that specularly reflected and diffusely reflected first illumination is entered into the imaging system and diffusely reflected second illumination is entered into the imaging system.

[0009] The computer-implemented method further includes determining information about specular reflection from a surface of the subject by comparing the first imaging data and the second imaging data.

[0010] Several embodiments relating to the first aspect are described below.

[0011] In some embodiments, the time frame during which the first imaging data is acquired at least partially overlaps with the time frame during which the second imaging data is acquired.

[0012] In some embodiments, the first and second imaging data are acquired simultaneously.

[0013] In some embodiments, the imaging system polarizer is configured to admit the reflected first and second illuminations into the imaging system having electric field components parallel to a polarization axis of the imaging system polarizer, and the imaging system polarizer is further configured to attenuate the reflected first and second illuminations having electric field components perpendicular to the polarization axis.

[0014] In some embodiments, the information about the specular reflectance is indicative of a gloss level of the subject's skin, the gloss level being determined by calculating a difference between intensity information in the first imaging data and intensity information in the second imaging data.

[0015] In some embodiments, the first and second color filters are part of a color filter array. The color filter array is configured to enable at least one imaging device of the imaging system to acquire first imaging data in the first spectral band and second imaging data in the second spectral band. The computer-implemented method further comprises extracting the first imaging data separately from the second imaging data from raw imaging data acquired by the at least one imaging device.

[0016] In a second aspect, a tangible machine-readable medium is described comprising instructions that, when executed by a processing circuit, cause the processing circuit to perform a method of the first aspect or any related embodiment.

[0017] In a third aspect, an apparatus is described that includes a processing circuit. The processing circuit includes a receiving module and a determining module.

[0018] The receiving module is configured to receive first and second imaging data acquired by the imaging system of a subject illuminated by a first illumination in a first spectral band and a second illumination in a second spectral band having a different spectral content than the first spectral band, the second illumination incident on the subject being polarized.

[0019] The received first imaging data is acquired within a first spectral band as a result of a first color filter of the imaging system allowing at least a portion of the first spectral band to enter the imaging system and preventing at least a portion of the second spectral band from entering the imaging system, such that a majority of intensity information in the first imaging data acquired within the first spectral band is derived from the first illumination.

[0020] The received second imaging data is acquired within a second spectral band as a result of a second color filter of the imaging system allowing at least a portion of the second spectral band to enter the imaging system and preventing at least a portion of the first spectral band from entering the imaging system, such that a majority of the intensity information in the second imaging data acquired within the second spectral band is derived from the second illumination.

[0021] Depending on the polarization states of the reflected first and second illumination received by the imaging system after reflection from the subject's surface, the first and second illumination are entered into the imaging system via an imaging system polarizer of the imaging system, such that specularly reflected and diffusely reflected first illumination is entered into the imaging system and diffusely reflected second illumination is entered into the imaging system.

[0022] The determination module is configured to determine information regarding specular reflection from a surface of the subject by comparing the first imaging data and the second imaging data.

[0023] Several embodiments relating to the third aspect are described below.

[0024] In some embodiments, the determination module is configured to determine a measure of skin radiance of the subject based on a comparison between the first imaging data and the second imaging data.

[0025] In some embodiments, the apparatus further comprises an imaging system and / or an illumination system configured to provide the first and second illumination.

[0026] In some embodiments, the imaging system polarizer is configured to allow the reflected first and second illuminations to enter the imaging system with electric field components parallel to a polarization axis of the imaging system polarizer, and the imaging system polarizer is further configured to prevent the reflected first and second illuminations from entering the imaging system with electric field components perpendicular to the polarization axis.

[0027] In some embodiments, the illumination system comprises an illumination system polarizer configured to polarize the second illumination directed at the subject, the polarization axis of the imaging system polarizer being orthogonal to the polarization axis of the illumination system polarizer.

[0028] In some embodiments, the illumination system is configured such that the first illumination directed at the subject is unpolarized, or the illumination system comprises an additional illumination system polarizer configured to polarize the first illumination directed at the subject such that the polarization state of the first illumination directed at the subject is orthogonal to the polarization state of the second illumination directed at the subject.

[0029] In some embodiments, the illumination system is configured to direct the first and second illuminations toward the subject such that both specular and diffuse reflected components of the first and second illuminations reflected from a surface of the subject are directed into the imaging system for internal ingress according to the polarization states of the reflected first and second illuminations.

[0030] In some embodiments, the imaging system comprises at least one imaging device and a light filter array, the light filter array comprising a first color filter and a second color filter.

[0031] The optical filter array is configured to pass at least a portion of the first spectral band into the imaging system to a first set of pixels of the at least one imaging device such that a majority of intensity information in first imaging data acquired within the first spectral band is derived from the first illumination.

[0032] The optical filter array is further configured to pass at least a portion of the second spectral band into the imaging system to a second, different set of pixels of the at least one imaging device such that a majority of intensity information in second imaging data acquired within the second spectral band is derived from the second illumination.

[0033] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiment(s) described hereinafter.

[0034] Example embodiments of the present invention will now be described, by way of example only, with reference to the following drawings: [Brief description of the drawings]

[0035] [Figure 1] FIG. 1 illustrates a method for determining specific information about a surface of a subject according to one embodiment. [Diagram 2] FIG. 1 is a schematic diagram of a system for determining specific information about a surface of a subject, according to one embodiment. [Diagram 3] FIG. 1 is a schematic diagram of a system for determining specific information about a surface of a subject, according to one embodiment. [Figure 4] 4A-4C show exemplary optical parameters of certain components of the system of FIG. 3. [Diagram 5] FIG. 1 is a schematic diagram of a representation of a method for determining specific information about a surface of a subject according to one embodiment. [Figure 6] FIG. 1 illustrates a method for determining specific information about a surface of a subject according to one embodiment. [Figure 7] FIG. 1 is a schematic diagram of a machine-readable medium for determining specific information about a surface of a subject in accordance with one embodiment. [Figure 8] FIG. 1 is a schematic diagram of an apparatus for determining specific information about a surface of a subject according to one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0036] According to skin beauty assumptions, it is desirable for skin to appear to have a natural, luminous sheen without appearing oily. The appearance of skin depends on how light interacts with the skin. The level of apparent sheen depends on the underlying surface and subsurface reflectance resulting from the various angles of light incident on the skin.

[0037] Measuring skin radiance is of interest in efficacy testing of certain skin care solutions, for example, as skin radiance is related to skin beauty and influences an individual's self-confidence and / or achievement of a particular appearance.

[0038] Skin care products, such as makeup and moisturizers, when applied to the skin, affect the apparent radiance of the skin. Additionally, some devices, such as skin cleansing devices, shavers, exfoliators, hydration devices, skin stimulators (e.g., mechanical, electrical, optical) or any other device that changes the appearance of the skin (e.g., for cosmetic or other reasons), also affect the apparent radiance of the skin. Skin care products and / or devices are used as part of a personal care regimen.

[0039] Certain personal care regimens involve characterizing skin radiance (e.g., before and / or after the personal care regimen). A user interested in characterizing their skin (e.g., to determine skin radiance) uses certain embodiments described herein to determine information useful for such characterization. This information can be useful in evaluating the personal care regimen.

[0040] 1 illustrates a method 100 (e.g., a computer-implemented method) for determining certain information about a surface of a subject (e.g., a user's skin). The method 100 is implemented by a user device or a computer, such as a server or cloud-based service (communicatively coupled to the user device). An example of a user device includes a smart device, such as a smartphone, tablet, smart mirror, or any other device capable of processing imaging data as described below.

[0041] The method 100 includes, at block 102, receiving first and second imaging data.

[0042] As described in more detail below, the first and second imaging data are acquired by the imaging system (before being received according to block 102 of method 100). Imaging data refers to information, such as pixel intensity information, derived from at least one image acquired by the imaging system of a subject illuminated by a first illumination and a second illumination. The first illumination is in a first spectral band and the second illumination is in a second spectral band. The second spectral band includes a different spectral content than the first spectral band. The first spectral band may or may not overlap with the second spectral band. Further details of the first and second spectral bands are described in more detail below. The second illumination incident on the subject is polarized (e.g., linearly polarized).

[0043] The received first imaging data is acquired within a first spectral band as a result of a first color filter of the imaging system allowing at least a portion of the first spectral band to enter the imaging system and preventing at least a portion of the second spectral band from entering the imaging system, such that a majority of intensity information in the first imaging data acquired within the first spectral band is derived from the first illumination.

[0044] The received second imaging data is acquired within a second spectral band as a result of a second color filter of the imaging system allowing at least a portion of the second spectral band to enter the imaging system and preventing at least a portion of the first spectral band from entering the imaging system, such that a majority of the intensity information in the second imaging data acquired within the second spectral band is derived from the second illumination.

[0045] Thus, the imaging system acquires both first imaging data and second imaging data, and the second imaging data is distinct from the first imaging data in that the reflectance information is distinct between the first and second imaging data.

[0046] The first imaging data corresponds to imaging performed in a first spectral band. The second imaging data corresponds to imaging performed in a second spectral band. Depending on the spectral overlap between the channels used to acquire the first and second imaging data and the spectral overlap between the first and second spectral bands, the first imaging data may or may not include information derived from the second illumination, and vice versa.

[0047] Imaging system embodiments described herein provide ways to ensure that a majority of the intensity information in first imaging data acquired in a first spectral band (e.g., at least 50% of the sum of the pixel intensity values) is derived from a first illumination. Similarly, such embodiments provide ways to ensure that a majority of the intensity information in second imaging data acquired in a second spectral band (e.g., at least 50% of the sum of the pixel intensity values ​​from an image) is derived from a second illumination.

[0048] In other words, at least 50% (i.e., a majority) of the total intensity information registered in the first imaging data (e.g., the sum of pixel intensity values ​​in the first imaging data) results from a first illumination (e.g., "red" light) being incident on the imaging system and detected by pixels of an imaging device of the imaging system. Correspondingly, less than 50% (i.e., a "minority") of the total intensity information registered in the first imaging data (e.g., the sum of pixel intensity values ​​in the first imaging data) results from a second illumination (e.g., "blue" light) being incident on the imaging system and detected by pixels of an imaging device of the imaging system. Corresponding logic is applied to the second imaging data.

[0049] The first and second illumination are entered into the imaging system via an imaging system polarizer of the imaging system according to the polarization states of the reflected first and second illumination received by the imaging system after reflection from the subject's surface such that specularly and diffusely reflected first illumination is entered into the imaging system and diffusely reflected second illumination is entered into the imaging system.

[0050] As described in more detail below, the first and second illumination incident (i.e., directed) on the subject has a particular initial polarization state (e.g., polarized or unpolarized). Upon reflection from the surface (and / or subsurface) of the subject, this polarization state may or may not be preserved. For example, diffusely reflected illumination is unpolarized (or randomly polarized) regardless of the polarization state of the illumination incident on the surface (e.g., surface "roughness" at least partially randomizes the polarization state after reflection). However, if the illumination incident on the surface is initially polarized, the specularly reflected illumination will at least partially maintain the polarization state of the illumination incident on the surface. Unpolarized illumination incident on the surface will remain unpolarized after specular reflection. In some cases, however, unpolarized illumination incident on the surface will be at least partially polarized after reflection depending on the angle of incidence.

[0051] The imaging system is configured such that specularly and diffusely reflected illumination is admitted or excluded from the imaging system depending on its polarization state upon incidence on the imaging system (i.e., after reflection from a surface).

[0052] The method 100 further includes, at block 104, determining information about specular reflection from a surface of the subject by comparing the first imaging data and the second imaging data.

[0053] As described above, the first imaging data includes information about a first illumination that is both specularly and diffusely reflected. The second imaging data includes information about a second illumination that is diffusely reflected. Specular reflection indicates a level of apparent glossiness, while diffuse reflection obscures such specular reflection. By comparing the first imaging data and the second imaging data, it is possible to determine information about the specular reflection, and therefore the level of glossiness, since the comparison separates the specular reflection information from the diffuse reflection information.

[0054] Thus, in some embodiments, the information about the specular reflection is indicative of a gloss level of the subject's skin. The gloss level is determined by calculating a difference between the intensity information in the first imaging data and the intensity information in the second imaging data. In similar terms, the result of the comparison between the first imaging data and the second imaging data corresponds to a measurement of the glossiness of the subject's skin.

[0055] Certain embodiments described herein facilitate visualization of skin tone (e.g., for the entire face) in a relatively inexpensive manner. For example, a user device, such as a smartphone with an imaging system with an imaging device, can acquire the first and second imaging data. In some cases, additional equipment, such as a hardware module (e.g., with a polarizer and / or an illumination system) that can be coupled to or used in situ with the user device, is used to facilitate acquisition of the first and second imaging data (e.g., to introduce the first and second illuminations according to their polarization states and / or to provide the first and second illuminations as mentioned in method 100). In other cases, a dedicated device comprises hardware and corresponding functionality to facilitate acquiring the first and second imaging data, providing the first and second illuminations, and / or determining the information mentioned in method 100. Furthermore, certain embodiments described herein facilitate providing precise and / or accurate measurement and tracking / monitoring of skin tone over time. In some cases, certain embodiments described herein provide a straightforward / low complexity way to accurately determine specular information. Certain embodiments described herein provide a highly accurate way of distinguishing between specular and diffuse information in imaging data, potentially without the need to acquire separate images at different times and / or without the need to use complex lighting or imaging system setups to acquire the images. The light beam containing both specular and diffuse information follows a common path / same angle in the imaging system, which reduces complexity and / or allows for more accurate estimation of the specular contribution. In some cases, certain embodiments described herein provide a relatively low-complexity (and cost-effective) solution to the problem of measuring a subject's shine, for example through the use of a polarizer in the imaging system, as described below.

[0056] 2 illustrates a system 200 for determining certain information about a surface of a subject according to certain embodiments. System 200 at least partially implements certain methods described herein, such as method 100 above. In some embodiments, certain blocks of system 200 are omitted.

[0057] The system 200 is used by a subject 202 and comprises an imaging system 204 and an illumination system 206. The imaging system 204 is used to acquire imaging data as referred to in the method 100. The illumination system 206 is configured to provide first and second illumination 206a, 206b. The imaging system 204 and / or the illumination system 206 are implemented by at least one device, such as a user device. Thus, in some embodiments, separate devices comprise the imaging system 204 and the illumination system 206, while in other embodiments, the same device comprises the imaging system 204 and the illumination system 206.

[0058] The system 200 further comprises a computer 208 (e.g., with processing circuitry implemented by a device or a server or cloud-based service to perform certain methods described herein). As such, the computer 208 is communicatively coupled to the imaging system 204 and / or the lighting system 206 for transmitting and / or receiving data to and from these systems. This data is processed by the processing circuitry of the computer 208 and / or stored in a memory (e.g., of the computer 208 or accessible to the processing circuitry of the computer 208). In some embodiments, the computer 208 controls the operation of the imaging system 204 and / or the lighting system 206. In some embodiments, the computer 208 comprises controls for controlling illumination parameters (e.g., operational parameters for the lighting system 206) and / or detection parameters (e.g., operational parameters for the imaging system 204) and for storing and / or processing captured images or videos.

[0059] As shown in FIG. 2, both the first illumination 206a (solid line) and the second illumination 206b (dashed line) are directed by the illumination system 206 to the surface of the subject 202. The first and second illuminations 206a, 206b incident on the subject are then reflected (e.g., specularly and / or diffusely reflected), which directs at least a portion of the reflected illuminations 206a, 206b towards and into the imaging system 204. As shown in FIG. 2, the angle between the incident light and the reflected light depends on the respective positions of the imaging system 204 and the illumination system 206 relative to the subject 202. Although the operation of the system 200 is not strongly angle-dependent, different angles affect how much the illuminations 206a, 206b are specularly or diffusely reflected. In one embodiment, the angle of incidence is as close as possible to normal incidence (e.g., less than 10 degrees), but it should be understood that other angle ranges may also be used depending on the configuration of the imaging system 204 and the illumination system 206.

[0060] FIG. 3 shows different views (a) and (b) of a system 300 for determining certain information about a surface of a subject according to certain embodiments. View (a) is a side view of the system 300 in a plane including the optical axis. View (b) is a front view of certain components of the system 300 in a plane perpendicular to the optical axis. The components of features of the system 300 are schematic and may be provided in any suitable arrangement other than that shown in FIG. 3. Reference numbers for features in the system 300 that have the same or similar functions as corresponding features in the system 200 are incremented by 100. The system 300 comprises certain corresponding features of the system 200 (i.e., an imaging system 304, an illumination system 306, and a computer 308). The system 300 at least partially implements certain methods described herein, such as the method 100.

[0061] Next, the imaging system 304 will be described.

[0062] The imaging system 304 includes a polarizing device (i.e., "imaging system polarizer" 310). The imaging system polarizer 310 is configured to admit reflected first and second illuminations 306a, 306b (i.e., reflected from the surface of the subject 202) with electric field components parallel to the polarization axis of the imaging system polarizer 310 into the imaging system and to attenuate reflected first and second illuminations 306a, 306b with electric field components perpendicular to the polarization axis. The first and second illuminations 306a, 306b that are attenuated upon admission to the imaging system 304 are attenuated by absorption or reflection by the imaging system polarizer 310. "Polarization axis" refers to the electric field vector direction selected for admission to the imaging system 304. For example, if the electric field vector of the illuminations 306a, 306b is parallel to the polarization axis, the polarizing device admits this illumination 306a, 306b into the imaging system 403.

[0063] 3, the polarization axis of the imaging system polarizer 310 is the same for both the first and second illuminations 306a, 306b. Thus, in some cases, a single or "common" imaging system polarizer 310 is used to control the incidence of both the reflected first and second illuminations 306a, 306b (depending on their polarization) into the imaging system 304. The use of a common imaging system polarizer 310 provides a simple / low-cost arrangement for controlling the incidence of the reflected first and second illuminations 306a, 306b.

[0064] In some cases, the imaging system polarizer 310 comprises a (linear) polarizing device, such as a polarizing filter, that allows (at least partial) transmission of the first and second illuminations 306a, 306b when the illuminations 306a, 306b contain a non-zero electric field (vector) component parallel to the polarization axis (also called the "transmission axis") of the polarizing filter. The polarizing filter attenuates the transmission of the first and second illuminations 306a, 306b when they contain a non-zero electric field (vector) component perpendicular to the transmission axis of the polarizing filter.

[0065] In some cases, the imaging system polarizer 310 comprises a polarizing device, such as a polarizing beam splitter (PBS, not shown), that can transmit or reflect illumination depending on the polarization state of the incident illumination. Such a polarizing device is configured to admit illumination of a particular polarization state into the imaging system 304 depending on the polarization state (e.g., either polarized illumination reflected from the PBS or polarized illumination transmitted by the PBS depending on the configuration of the PBS). In either case, the polarization axis of the PBS is oriented in a manner to select which polarization state is admitted to the imaging system 304.

[0066] Thus, the portion of the illumination 306a, 306b that has a zero electric field vector component perpendicular to the polarization axis of the imaging system polarizer 310 (and a non-zero component parallel to the polarization axis) is admitted to the imaging system 304 with little or no attenuation by the imaging system polarizer 310.

[0067] However, the portion of the illumination 306a, 306b that has a zero electric field vector component parallel to the polarization axis of the imaging system polarizer 310 (and a non-zero component perpendicular to the polarization axis) is attenuated (potentially completely attenuated) by the imaging system polarizer 310. Thus, the level of attenuation depends on the ratio of the electric field vector component parallel to the polarization axis of the imaging system polarizer 310 to the electric field vector component perpendicular to the polarization axis.

[0068] Thus, the imaging system polarizer 310 admits and / or attenuates the first and second illumination 306a, 306b reflected from the surface of the subject 302 depending on the polarization state of the reflected illumination 306a, 306b directed into the imaging system 304.

[0069] An optical axis ("z") is defined between the imaging system 304 and the subject 302. The optical axis is also referred to as the "imaging axis." Relative to the optical axis z, the imaging system polarizer 310 has a "horizontal" polarization axis in the "x" direction, indicated by a point adjacent the imaging system polarizer 310 (the corresponding "y" direction is shown vertically in FIG. 3). Thus, illumination 306a, 306b having a non-zero electric field component in the x-axis is introduced into the imaging system 304 by the imaging system polarizer 310, but may be attenuated depending on whether there is a non-zero electric field component perpendicular to the polarization axis of the imaging system polarizer 310. The coordinate system shown in FIG. 3 and elsewhere in this disclosure is provided to aid in explanation, and other coordinate systems may be used.

[0070] In this embodiment, the imaging system polarizer 310 is configured to couple the first and second illuminations 306a, 306b to the imaging system 304 according to the polarization states of the reflected first and second illuminations 306a, 306b received by the imaging system 304, such that after reflection from the surface of the subject 302, a specularly and diffusely reflected first illumination 306a is coupled to the imaging system 304 and a diffusely reflected second illumination 306b is coupled to the imaging system 304. The illuminations 306a, 306b coupled to (i.e., after coupling to) the imaging system polarizer 310 have a zero electric field component perpendicular to the polarization axis of the imaging system polarizer 310 (assuming a perfectly efficient polarizer, which is unlikely in practice).

[0071] In some embodiments, the specularly and diffusely reflected first illumination 306a is attenuated depending on the polarization state of the reflected first illumination 306a incident on the subject 302. For example, if the polarization state of the reflected first illumination 306a is random, approximately 50% of the total reflected first illumination 306a is incident (the remaining 50% is attenuated by absorption or reflection). However, if the reflected first illumination 306a is not randomly polarized, the level of attenuation will vary depending on the angle between the polarization state and the polarization axis of the first illumination 306a. As a result, the first imaging data includes information about the first illumination 306a that has been both specularly and diffusely reflected.

[0072] Furthermore, the specularly reflected second illumination 306b is likely to be almost completely attenuated if it has a zero electric field vector component parallel to the polarization axis of the imaging system polarizer 310. The polarized second illumination 306b (before reflection) has an electric field component perpendicular to the polarization axis of the imaging system polarizer 310. If the polarization state is maintained (and not substantially rotated) after reflection (i.e., not randomized by surface roughness), the reflected second illumination 306b has a zero electric field component parallel to the polarization axis of the imaging system polarizer 310. In this case, the second illumination 306b is consequently attenuated so much that little or no second illumination 306b is admitted by the imaging system polarizer 310. In the case of surface roughness or low gloss, some diffuse reflection of the second illumination 306b occurs, which at least partially randomizes the polarization state of the (originally polarized) second illumination 306b. Thus, at least a portion of the diffusely reflected second illumination 306b includes a non-zero electric field component aligned with the polarization axis, allowing at least attenuated entry of said diffusely reflected second illumination 306b. Thus, the second imaging data includes information regarding the diffusely reflected second illumination 306b (but not regarding the specularly reflected second illumination 306b).

[0073] Thus, the imaging system polarizer 310 is used to admit and / or attenuate illuminations 306a, 306b having a particular polarization state, i.e., the orientation of the polarization axis of the imaging system polarizer 310 is such that a particular portion of the reflected first and second illuminations 306a, 306b is admitted or attenuated depending on the polarization state of the reflected first and second illuminations 306a, 306b.

[0074] As highlighted above, the reflected first and second illuminations 306a, 306b have multiple polarization states due to the polarization states of different portions of the reflected first and second illuminations 306a, 306b being modified to different degrees after reflection from different surface types. For example, polarized second illumination 306b incident on a rough surface will be randomly polarized after reflection, whereas polarized second illumination 306b incident on a smooth and shiny surface will maintain its polarization state after reflection (i.e., the originally polarized illumination will remain polarized after reflection).

[0075] In some embodiments, the initially unpolarized (or randomly polarized) first illumination 306a remains unpolarized after reflection from a rough surface. In some embodiments, the initially unpolarized (or randomly polarized) first illumination 306a becomes at least partially polarized after reflection from a smooth or shiny surface. In some embodiments, the first illumination 306a is polarized with an initial polarization state that is orthogonal to the polarization state of the second illumination 306b.

[0076] Thus, for an unpolarized first illumination 306a and a polarized second illumination 306b incident on a smooth and / or shiny subject 302, the first imaging data (corresponding to imaging using the first illumination 306a) contains information about the first illumination 306a that has been both specularly and diffusely reflected. The reflected second illumination 306b entering the imaging system is substantially attenuated by the imaging system polarizer 310 due to its zero or near-zero electric field component parallel to the polarization axis of the imaging system polarizer 310, so that the pixel intensity levels registered for the second imaging data (corresponding to imaging using the second illumination 306b) will be much lower (i.e., darker) than the pixel intensity levels registered for the first imaging data.

[0077] Thus, information regarding the specular reflection information is determined by calculating the difference between the first imaging data (containing both specular and diffuse reflection information) and the second imaging data (containing diffuse reflection information). In the above case, the shiny surface results in a low level of diffuse reflection and a relatively high level of specular reflection.

[0078] In the case of unpolarized first illumination 306a and polarized second illumination 306b incident on the rough subject 302, the first imaging data (corresponding to imaging using the first illumination 306a) includes information about the first illumination 306a that has undergone both specular and diffuse reflection. The pixel intensity levels registered for the second imaging data (corresponding to imaging using the second illumination 306) are not darkened (unlike in the previous case) because the reflected second illumination 306 entering the imaging system 304 is not substantially attenuated by the imaging system polarizer 310 (due to its non-zero electric field components parallel to the polarization axis of the imaging system polarizer 310 resulting from the reflected second illumination 306b, including the diffuse reflection, which may at least partially be randomly polarized).

[0079] Thus, information about specular reflection is determined by calculating the difference between the first imaging data (containing both specular and diffuse reflection information) and the second imaging data (containing diffuse reflection information). In the above case, the rough surface results in a high level of diffuse reflection and a relatively low level of specular reflection.

[0080] Thus, the ratio of (or the difference between) pixel intensity values ​​in the first and second imaging data indicates the relative amount of specular and diffuse reflection from the surface. If the second imaging data is relatively "darker" than the first imaging data (with lower registered pixel intensity values), this is indicative of a shiny surface. However, if the second imaging data is relatively "lighter" than the first imaging data (with relatively higher registered intensity values ​​compared to the "shiny" case), this is indicative of a rough surface.

[0081] In some cases, a ratio or difference between pixel intensity values ​​in the first and second imaging data provides a quantitative measure of specular reflectance.

[0082] In some embodiments, the imaging system 304 comprises (at least one) first color filter and (at least one) second color filter. The color filter array 312 is configured to enable at least one imaging device 314 of the imaging system 304 to acquire first imaging data in a first spectral band (through the first color filter) and second imaging data in a second spectral band (through the second color filter). The color filter array 312 is provided as part of the imaging device 314 itself (e.g., a Bayer filter or another type of filter layer) or as a separate component.

[0083] In one embodiment, the color filter array 312 is aligned with the pixels (not shown) of the imaging device 314 such that each filter "cell" or "channel" of the array 312 is aligned with a corresponding pixel of the imaging device 314. For example, a Bayer filter comprises red, green, and blue bandpass filter "cells" aligned with pixels of the imaging device 314. Thus, some pixels of the imaging device 314 register "red" light, some pixels register "green" light, and the remaining pixels register "blue" light. In one embodiment, the "red" pixels provide the first imaging data and the "green" pixels provide the second imaging data. In another embodiment, the "red" pixels provide the first imaging data and the "blue" pixels provide the second imaging data. Any other combination of colors associated with the pixels may provide the first and second imaging data.

[0084] Thus, in some embodiments, the imaging system 304 comprises at least one imaging device 314 and an optical filter array 312. The optical filter array 312 comprises a first color filter and a second color filter. The optical filter array 312 is configured to pass at least a portion of a first spectral band into the imaging system 304 to a first set of pixels of the at least one imaging device 314 such that a majority of the intensity information in the first imaging data acquired in the first spectral band is derived from the first illumination 306a. The optical filter array 312 is further configured to pass at least a portion of a second spectral band into the imaging system 304 to a second, different set of pixels of the at least one imaging device 314 such that a majority of the intensity information in the second imaging data acquired in the second spectral band is derived from the second illumination 306a.

[0085] The imaging device 314 and / or computer 308 extract the first imaging data separately from the second imaging data from the raw imaging data acquired by the imaging device 314. For example, certain pixels of the imaging device 314 that correspond to a particular color (e.g., one of red, green, or blue) provide pixel intensity information as the "raw" data for the first imaging data. Certain other pixels of the imaging device 314 that correspond to another color (e.g., one of red, green, or blue) provide pixel intensity information as the "raw" data for the second imaging data.

[0086] In another embodiment, the imaging system 304 comprises a plurality (e.g., two or more) imaging devices (not shown). A first one of the imaging devices receives the first illumination but not the second illumination (e.g., via a suitable optical arrangement including suitable optical filters). A second one of the imaging devices receives the second illumination but not the first illumination. Thus, the first imaging device provides the first imaging data and the second imaging device provides the second imaging data.

[0087] According to the above-mentioned arrangement, the first and second imaging data may be acquired over the same or overlapping time periods. In other words, in some embodiments, the time frame during which the first imaging data is acquired at least partially overlaps with the time frame during which the second imaging data is acquired. This occurs when a particular pixel (or first imaging device) associated with the first imaging data performs imaging (e.g., acquires a frame) over a first time period, and the second imaging data performs imaging over a second time period that at least partially overlaps with the first time period.

[0088] In another embodiment, the imaging system 304 is configured to acquire the first and second imaging data simultaneously, for example, pixels configured to measure intensity levels for the first imaging data measure such intensity levels simultaneously as pixels configured to measure intensity levels for the second imaging data.

[0089] By acquiring the first and second imaging data over overlapping time periods and / or simultaneously, imaging data for determining information regarding the specular reflection is acquired in a relatively straightforward and inexpensive manner. For example, a user device such as a smartphone is configured to acquire the first and second imaging data in a straightforward manner by providing an imaging system polarizer in conjunction with the imaging device of the user device described above. In some cases, the imaging system polarizer 310 is a component of the user device itself, or in other cases, it is a separate component.

[0090] Because a subject's surface (e.g., a user's face) moves voluntarily and / or involuntarily, acquiring first and second imaging data over overlapping time periods and / or simultaneously reduces errors in determining information about specular reflection. For example, if the first and second imaging data are acquired simultaneously, the same pixels are likely to register information from the same portion of the user's facial surface. Additionally, in some cases, gloss measurements are taken of multiple views of the face to cover multiple angles. The embodiments described herein facilitate acquiring a sequence of frames (e.g., a video or image sequence) using an imaging device capable of simultaneously spectrally separating the first and second imaging data.

[0091] The spectral gating of the first and second illuminations, and the polarization dependence of the illuminations 306a, 306b incident on the imaging system 304, allows the first imaging data to be distinguished from the second imaging data. Thus, the configuration of the imaging system 304 and the illumination system 306 provides a way to easily distinguish between reflected illumination 306a, 306b containing specular information and reflected illumination 306a, 306b containing diffuse information, and thus determine the specular information.

[0092] Next, the lighting system 306 will be described.

[0093] The illumination system 306 comprises a first illumination source 316 (e.g., at least one light emitting diode (LED) or other type of light source) configured to provide a first illumination 306a. In this embodiment, the first illumination 306a is unpolarized and directed towards the subject 302. The illumination system 306 comprises a second illumination source 318 (e.g., at least one light emitting diode (LED) or other type of light source) configured to provide a second illumination 306b. The second illumination 306a is directed towards the subject 302 in a similar manner as the first illumination 306b. In this regard, the first illumination source 316 and the second illumination source 318 may be adjacent to each other or in any suitable arrangement for illuminating the subject 302.

[0094] The illumination system 306 includes an illumination system polarizer 320 configured to polarize the second illumination 306b directed toward the subject 302. The polarization axis of the imaging system polarizer 310 is orthogonal (e.g., relative to the optical axis) to the polarization axis of the illumination system polarizer 320. In similar terms, the polarization states incident or transmitted by the imaging system polarizer 310 and the illumination system polarizer 320 are orthogonal to each other (or the two orthogonal polarization states represent diagonally opposite points on the Poincaré sphere). As shown in FIG. 3, the imaging system polarizer 310 has a polarization axis in the x-direction and the illumination system polarizer 320 has a polarization axis in the y-direction.

[0095] In some embodiments, the illumination system 306 is configured to direct the first and second illuminations 306a, 306b toward the subject 302 such that both specular and diffuse reflection components of the first and second illuminations 306a, 306b reflected from a surface of the subject 302 are directed into the imaging system for internal ingress depending on the polarization states of the reflected first and second illuminations 306a, 306b.

[0096] As discussed above, the angle of incidence varies depending on the configuration. In a possible configuration, the illumination system 306 includes first and second illumination sources 316, 318 positioned adjacent the imaging system 304 such that the angle of incidence is as close as possible to normal incidence. In a possible configuration, multiple first and second illumination sources 316, 318 are positioned around the imaging system 304 (e.g., in a ring, concentrically) such that the subject 302 is uniformly illuminated from multiple angles.

[0097] In the embodiment shown in Figure 3, the illumination system 306 is configured such that the first illumination 306a directed towards the subject 302 is unpolarized. However, in some embodiments, the illumination system 306 comprises an additional illumination system polarizer (not shown) configured to polarize the first illumination 306a directed towards the subject 302 such that the polarization state of the first illumination 306a incident on the subject 302 is orthogonal to the polarization state of the second illumination 306b incident on the subject. In other words, the additional illumination system polarizer has its polarization axis oriented along the x-axis in Figure 3.

[0098] Figure 4 shows example optical parameters of certain components of the system 300 of Figure 3 in graphical form of percent transmission as a function of wavelength (for a solid line), overlaid with dotted lines corresponding to the spectral content of the first and second illumination sources 316, 318 referenced in Figure 3. The reference numbers of the features referenced in Figure 3 are incremented by 100.

[0099] In case #1, the first illumination 406a (dotted line) includes a spectral content in a first spectral band centered around 650 nm (nanometers), i.e., toward the "red" portion of the spectrum. The full width at half maximum spectral width of the first spectral band is around 60 nm. The color filter array 312 in FIG. 3 comprises different "cells" for admitting / passing the first spectral band. In FIG. 4, the spectral admission band for admitting the first illumination 406a is shown by a solid line 412a. The spectral admission band is mainly in the "red" to "infrared" portion of the spectrum (e.g., between approximately 570 and 900 nm). The first spectral band of the first illumination 406a substantially overlaps with the spectral admission band 412a. Thus, a cell in the color filter array 312 of FIG. 3 that corresponds to the spectral light entrance band 412a allows the first illumination 406a to enter such that it is detected by the corresponding pixel (e.g., to reduce detection of illumination in other spectral bands).

[0100] The second illumination 406b (dotted line) includes spectral content in a second spectral band centered around 450 nm, i.e., toward the "blue" portion of the spectrum. The full width at half maximum spectral width of the second spectral band is around 50 nm. The spectral entry band for admitting the second illumination 406b is shown by the solid line 412b. The spectral entry band is primarily in the "bluer" portion of the spectrum (e.g., between <400 and 550 nm). The second spectral band of the second illumination 406b substantially overlaps with the spectral entry band 412b. Thus, the cells in the color filter array 312 of FIG. 3 corresponding to the spectral entry band 412b admit the second illumination 406b so that it is detected by the corresponding pixel (e.g., to reduce detection of illumination in other spectral bands, such as the first illumination 406a).

[0101] In case #2 of FIG. 4, the first illumination 406a and the spectral entry band 412a are the same as in case #1. However, the second illumination 406b (dotted line) includes a spectral content in a second spectral band centered around 530 nm, i.e., including the "green" portion of the spectrum. The full width at half maximum of the second spectral band is around 50 nm. The spectral entry band for entering the second illumination 406b is shown by the solid line 412b. The spectral entry band is mainly in the "green" portion of the spectrum (e.g., between approximately 450 and 610 nm). The second spectral band of the second illumination 406b substantially overlaps with the spectral entry band 412b. Thus, a cell in the color filter array 312 of FIG. 3 that corresponds to the spectral light entrance band 412b allows the second illumination 406b to enter such that it is detected by the corresponding pixel (e.g., to reduce detection of illumination in other spectral bands, such as the first illumination 406a).

[0102] Case #2 highlights how there may be some spectral overlap between the spectral entry bands and / or the first and second illuminations 406a, 406b when sufficient spectral differentiation is present as described below.

[0103] This spectral discrimination is obtained when the wavelength band (e.g., interval) of each illumination source 316, 318 includes 1) substantial spectral overlap (e.g., substantial overlap of the spectral content of illumination 406a, 406b with the corresponding spectral input band 412a, 412b) with the spectral sensitivity of the associated channel (e.g., a red light source has an associated red detection channel) of the imaging system 304, and 2) minimal spectral overlap with the spectral sensitivity of the non-associated channel. In similar terms, the majority of the intensity information in the first imaging data acquired in the first spectral band is derived from the first illumination 406a. Similarly, the majority of the intensity information in the second imaging data acquired in the second spectral band is derived from the second illumination 406b. There may be some "leakage" between the channels to some degree in case #1 and even more in case #2. However, the majority of the pixel intensity information is likely derived from the correct channel due to the spectral discrimination described above.

[0104] Case #1 refers to a scenario considered as having an ideal overlap (e.g., overlap measured as the ratio of the overlap area under the curve to the area of ​​the illumination source spectrum) between the first illumination source 406a and the corresponding detection channel defined by the spectral entrance band 412a. The same is true for the second illumination source 406b and the corresponding detection channel defined by the spectral entrance band 412a. Then, the spectral overlap between the first and second illuminations 406a, 406b and the non-associated detection channel (e.g., the second and first spectral entrance bands 412b, 412a, respectively) is minimal. For example, <10% of the total spectral width is overlapped. It may be observed that the spectral overlap between the spectrum of the first and second illuminations 406a, 406b and the spectrum of the non-associated detection channel is approximately equivalent to the error in the calculated specular reflection. In case #2, there is a larger degree of overlap compared to case #1, but there is sufficient spectral differentiation to facilitate the spectral gating operation.

[0105] The choice of one or more wavelengths for each of the first and second illuminations 406a, 406b is generally not important, other than being appropriate for the wavelength detection range of the associated detection channel. For example, instead of having the first illumination 406a including the red portion of the spectrum and the second illumination 406b including the blue or green portion of the spectrum, as in cases #1 and #2 above, another configuration may be selected. For example, the first illumination 406a may include the blue or green portion of the spectrum, and the second illumination 406b may include the green or red portion of the spectrum. Furthermore, a non-visible illumination source, for example ultraviolet or infrared, may be used for at least one of the illumination sources.

[0106] 3, and in some embodiments, the illumination system 306 comprises a first illumination unit (e.g., a first illumination source 316) for providing light illumination (e.g., directed towards the face of the subject 302). The first illumination unit provides the first illumination 406a by emitting in a first spectral emission band. The first spectral emission band is selected to substantially overlap with a first spectral detection band of the imaging unit (e.g., the imaging system 304).

[0107] The illumination system 306 comprises a second illumination unit (e.g., a second illumination source 318) for providing linearly polarized light (illumination) (e.g., directed towards the face of the subject 302). The second illumination unit provides the second illumination 406b by emitting in a second spectral emission band. The second spectral emission band is selected to substantially overlap with a second spectral detection band of the imaging unit.

[0108] A first linear polarizing filter (eg, illumination system polarizer 320) associated with the second illumination unit is oriented perpendicular to the polarization axis of the light entering the imaging unit.

[0109] The imaging unit includes a second linear polarizing filter (e.g., imaging system polarizer 310). The imaging unit is configured to admit at least two spectral detection bands. The imaging unit is configured to capture (e.g., image) the diffuse reflectance of the second illumination 406b reflected from a surface (e.g., face) of the subject 302. The imaging unit is further configured to capture the total reflectance (e.g., diffuse reflectance and specular reflectance) of the first illumination 406a reflected by the surface of the same subject. The wavelengths and bandwidths of the illuminations 406a, 406b are determined in combination with the spectral detection bands associated with the imaging unit.

[0110] A control unit (e.g., computer 308) is provided to control the illumination and detection parameters and to store and / or process the captured raw Bayer images or videos (e.g., receive the first and second imaging data and determine information regarding specular reflectance). As described above, specular reflectance is extracted from two separate channels from the raw Bayer image data.

[0111] Fig. 5 is a schematic diagram of a representation of a method for determining specific information about a surface of a subject according to an embodiment. Reference is made to the features of Fig. 3, the reference numbers of similar features being incremented by 200. This representation shows imaging data (represented by images) acquired by the procedure described above.

[0112] According to FIG. 5, specular and diffuse information is shown in the parallel "P" image, which is derived from the first incident illumination 306a. Diffuse information is shown in the cross "C" image, which is derived from the second incident illumination 306b. The total intensity is shown in the "P+C" image. The difference between the "P" and "C" images is shown in the "PC" image. The "PC" image thus corresponds to the specular image. Each of the images is derived by "de-bayering" or extracting the individual channels (i.e., red, blue, or green) incident by the corresponding pixels associated with the color filter array 512. FIG. 5 also shows a sequence of frames acquired at various angles of the specular image. The process for acquiring each frame is relatively straightforward (e.g., in some cases, one frame is acquired per angle, and only that one frame is needed to extract the specular information).

[0113] Some commercially available color cameras, such as those for mobile phones, have built-in Bayer filters (e.g., with green (two), red, and blue sub-filters). By utilizing a spectral gating procedure and polarization gating with orthogonal polarizers as described herein, cross- and parallel-polarized images of the face are acquired simultaneously without motion artifacts. Thus, cross- and parallel-polarized images can be acquired simultaneously, which reduces cost and / or complexity and allows for rapid acquisition and processing of frames, e.g., for multiple frames at different angles.

[0114] Due to the scattering and reflective properties of skin, the combination of a color camera and multiple illumination sources (at least one of which is polarized) results in less overlap in the Bayer spectra. This arrangement provides two images (e.g., corresponding to the first and second imaging data), one containing specular reflection and one containing no specular reflection at all (i.e., containing diffuse reflection instead).

[0115] The intensities recorded in the two images are distinguishable due to the different absorption, scattering and polarization properties of the skin (as well as the different intensities of the first and second illumination 306a, 306b provided by the illumination system 306). The sensitivity of the gloss measurement is refined by selecting appropriate wavelengths of light with a Bayer filter (or other type of color filter array 512 or other optical arrangement) and prior knowledge of the specular reflectance of the skin.

[0116] Additionally, certain methods described herein are extended by analyzing histograms of pixel intensity information in a "PC" image to quantitatively measure, for example, reduced greasiness following a personal care regimen. For example, a PC image histogram with a peak toward higher pixel intensity values ​​indicates a large amount of specular reflection, while a more even distribution of pixel intensity values ​​(or a peak toward lower pixel intensity values) indicates minimal specular reflection.

[0117] 6 illustrates a method 600 (e.g., a computer-implemented method) for determining certain information about a surface of a subject (e.g., a user's skin). The method 600 is implemented by a computer, such as a user device or a server or cloud-based service (e.g., communicatively coupled to a user device). An example of a user device includes a smart device, such as a smartphone, tablet, smart mirror, or any other device capable of processing imaging data as described below. Reference is made to FIG. 3 in the following description.

[0118] The method 600 includes blocks 102 and 104 of the method 100. The method 600 further includes, in block 602, extracting the first imaging data separately from the second imaging data from the raw imaging data acquired by the at least one imaging device 314. Block 602 is performed before block 102.

[0119] FIG. 7 illustrates a tangible, machine-readable medium 700 storing instructions 702 that, when executed by a processing circuit (e.g., at least one processor) 704, cause the processing circuit 704 to perform certain methods described herein, such as method 100, method 600, and / or related embodiments.

[0120] 8 illustrates an apparatus 800 that may be used to implement certain methods described herein, such as method 100, method 600, and / or related embodiments. The apparatus 800 includes modules whose functionality corresponds to certain features described in connection with the system 200 of FIG. 2, such as the computer 208. The apparatus 800 includes a processing circuit 802.

[0121] The processing circuit 802 comprises a receiving module 804 configured to receive first and second imaging data acquired by the imaging system 204 of the subject 202. The subject 202 is illuminated by a first illumination 206a in a first spectral band and a second illumination 206b in a second spectral band with a different spectral content than the first spectral band. The second illumination 206a incident on the subject is polarized.

[0122] The received first imaging data is acquired within a first spectral band as a result of a first color filter of the imaging system 204 allowing at least a portion of the first spectral band to enter the imaging system and preventing at least a portion of the second spectral band from entering the imaging system, such that a majority of the intensity information in the first imaging data acquired within the first spectral band is derived from the first illumination 206a.

[0123] The received second imaging data is acquired within a second spectral band as a result of a second color filter of the imaging system 204 allowing at least a portion of the second spectral band to enter the imaging system and preventing at least a portion of the first spectral band from entering the imaging system, such that a majority of the intensity information in the second imaging data acquired within the second spectral band is derived from the second illumination 206b.

[0124] The first and second illuminations are incident on the imaging system 204 via an imaging system polarizer 310 of the imaging system 204 according to the polarization states of the reflected first and second illuminations received by the imaging system after reflection from the subject's surface, such that specularly and diffusely reflected first illumination 206a is incident on the imaging system 204 and diffusely reflected second illumination 206b is incident on the imaging system 204.

[0125] The processing circuit 802 further comprises a determination module 806 configured to determine information regarding specular reflection from a surface of the subject 202 by comparing the first imaging data and the second imaging data.

[0126] In some embodiments, the apparatus 800 further comprises an imaging system 204. In some embodiments, the apparatus 800 further comprises an illumination system 206 configured to provide first and second illumination 206a, 206b.

[0127] In some embodiments, the determination module 806 is configured to determine a measure of the subject's skin radiance based on a comparison between the first imaging data and the second imaging data.

[0128] In some cases, any of the above-mentioned modules (e.g., the receiving module 804 and / or the decision module 806) include at least one dedicated processor (e.g., an application specific integrated circuit (ASIC) and / or a field programmable gate array (FPGA), etc.) for performing the functions of the module.

[0129] In some cases, the above modules (e.g., the receiving module 804 and / or the determining module 806) comprise at least one processor for executing instructions that cause the at least one processor to perform the functions of the module described above. In such examples, the instructions are stored on a machine-readable medium (not shown) accessible to the at least one processor. In some examples, the module itself comprises the machine-readable medium. In some examples, the machine-readable medium is separate from the module itself (e.g., at least one processor of the module is provided to communicate with the machine-readable medium to access the instructions stored therein).

[0130] Although certain methods have been described as computer-implemented, in some cases such methods refer to imaging methods in which an illumination system and / or an imaging system are used as part of the imaging method to enable acquisition of first and second imaging data (for use in implementing the computer-implemented method). Similarly, in some cases any apparatus for implementing such a computer-implemented method further comprises an illumination system and / or an imaging system to enable acquisition of the first and second imaging data.

[0131] This disclosure includes subject matter defined by the following numbered paragraphs:

[0132] Paragraph 1. Receiving imaging data acquired by an imaging system of a subject illuminated by a first illumination in a first spectral band and a second illumination in a second spectral band having a different spectral content than the first spectral band, the second illumination incident on the subject being polarized, and the imaging system: acquiring first imaging data within a first spectral band by allowing at least a portion of the first spectral band to enter the imaging system and preventing at least a portion of the second spectral band from entering the imaging system such that a majority of intensity information in the first imaging data acquired within the first spectral band is derived from the first illumination; acquiring second imaging data within a second spectral band by allowing at least a portion of the second spectral band to enter the imaging system and preventing at least a portion of the first spectral band from entering the imaging system such that a majority of intensity information in the second imaging data acquired within the second spectral band is derived from the second illumination; directing the first and second illuminations into the imaging system according to polarization states of the reflected first and second illuminations received by the imaging system after reflection from a surface of the subject, such that specularly and diffusely reflected first illumination is directed into the imaging system and diffusely reflected second illumination is directed into the imaging system. and determining information about specular reflection from a surface of the subject by comparing the first imaging data and the second imaging data; 13. A computer-implemented method comprising:

[0133] Paragraph 2. The computer-implemented method of Paragraph 1, wherein a time frame during which the first imaging data is acquired at least partially overlaps a time frame during which the second imaging data is acquired.

[0134] Paragraph 3. The computer-implemented method of Paragraph 2, wherein the imaging system is configured to acquire the first and second imaging data simultaneously.

[0135] Paragraph 4. The computer-implemented method of any one of paragraphs 1 to 3, wherein the imaging system includes an imaging system polarizer configured to allow reflected first and second illumination having electric field components parallel to a polarization axis of the imaging system polarizer to enter the imaging system, and to attenuate reflected first and second illumination having electric field components perpendicular to the polarization axis.

[0136] Paragraph 5. The computer-implemented method of any one of paragraphs 1 to 4, wherein the information regarding specular reflection indicates a gloss level of the subject's skin, and the gloss level is determined by calculating a difference between intensity information in the first imaging data and intensity information in the second imaging data.

[0137] Paragraph 6. The computer-implemented method of any one of paragraphs 1 to 5, wherein the imaging system comprises a color filter array configured to enable at least one imaging device of the imaging system to acquire first imaging data in a first spectral band and second imaging data in a second spectral band, and the computer-implemented method further comprises extracting the first imaging data separately from the second imaging data from raw imaging data acquired by the at least one imaging device.

[0138] Paragraph 7. A tangible, machine-readable medium comprising instructions that, when executed by a processing circuit, cause the processing circuit to perform the method of any one of paragraphs 1 to 6.

[0139] Paragraph 8. An apparatus having a processing circuit, the processing circuit comprising: 1. A receiving module configured to receive imaging data acquired by an imaging system of a subject illuminated by a first illumination in a first spectral band and a second illumination in a second spectral band having a different spectral content than the first spectral band, wherein the second illumination incident on the subject is polarized, and the imaging system is configured to: acquiring first imaging data within a first spectral band by allowing at least a portion of the first spectral band to enter the imaging system and preventing at least a portion of the second spectral band from entering the imaging system such that a majority of intensity information in the first imaging data acquired within the first spectral band is derived from the first illumination; acquiring second imaging data within a second spectral band by allowing at least a portion of the second spectral band to enter the imaging system and preventing at least a portion of the first spectral band from entering the imaging system such that a majority of intensity information in the second imaging data acquired within the second spectral band is derived from the second illumination; and coupling the first and second illuminations to the imaging system according to polarization states of the reflected first and second illuminations received by the imaging system after reflection from the subject's surface such that specularly and diffusely reflected first illumination is coupled to the imaging system and diffusely reflected second illumination is coupled to the imaging system. A receiving module configured to: and a determination module configured to determine information regarding specular reflection from a surface of the subject by comparing the first imaging data and the second imaging data.

[0140] Paragraph 9. The apparatus of Paragraph 8, wherein a result of the comparison between the first imaging data and the second imaging data corresponds to a measurement of the radiance of the subject's skin.

[0141] Paragraph 10. The apparatus of paragraphs 8 or 9, wherein the apparatus further comprises an imaging system and / or an illumination system configured to provide the first and second illumination.

[0142] Paragraph 11. The apparatus described in Paragraph 10, wherein the imaging system includes an imaging system polarizer configured to allow reflected first and second illumination having electric field components parallel to a polarization axis of the imaging system polarizer to enter the imaging system, and to prevent reflected first and second illumination having electric field components perpendicular to the polarization axis from entering the imaging system.

[0143] Paragraph 12. The apparatus described in Paragraph 11, wherein the illumination system includes an illumination system polarizer configured to polarize the second illumination directed at the subject, and a polarization axis of the imaging system polarizer is orthogonal to the polarization axis of the illumination system polarizer.

[0144] Paragraph 13. The apparatus of any one of paragraphs 10 to 12, wherein the illumination system is configured such that the first illumination directed at the subject is unpolarized or the illumination system comprises an additional illumination system polarizer configured to polarize the first illumination directed at the subject such that the polarization state of the first illumination directed at the subject is orthogonal to the polarization state of the second illumination directed at the subject.

[0145] Paragraph 14. An apparatus described in any one of paragraphs 10 to 13, wherein the illumination system is configured to direct the first and second illuminations toward the subject such that both specular and diffuse reflected components of the first and second illuminations reflected from a surface of the subject are directed into the imaging system for internal entry according to the polarization states of the reflected first and second illuminations.

[0146] Paragraph 15. An imaging system comprising at least one imaging device and an optical filter array, the optical filter array comprising: passing at least a portion of a first spectral band into the imaging system through a first set of pixels of at least one imaging device such that a majority of intensity information in first imaging data acquired within the first spectral band is derived from the first illumination; passing at least a portion of the second spectral band into the imaging system through a second, different set of pixels of the at least one imaging device such that a majority of intensity information in second imaging data acquired within the second spectral band is derived from the second illumination; 15. The apparatus of any one of paragraphs 10 to 14, configured to:

[0147] While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description is by way of illustration or example and is not to be considered as restrictive. The invention is not limited to the disclosed embodiments.

[0148] One or more features described in one embodiment may be combined with or substituted for features described in another embodiment, for example, the methods 100, 600 of Figures 1 and 6 may be modified based on features described in connection with the systems 200, 300, machine-readable medium 700, and / or apparatus 800 of Figures 2 and 3, and vice versa.

[0149] Embodiments of the present disclosure may be provided as a method, a system, or a combination of machine-readable instructions and processing circuitry. Such machine-readable instructions may be contained on a non-transitory machine (e.g., computer) readable storage medium (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-readable program code therein or thereon.

[0150] The present disclosure will be described with reference to flowcharts and block diagrams of methods, devices and systems according to embodiments of the present disclosure. Although the above flowcharts show a particular order of execution, the order of execution may differ from that shown. Blocks described in connection with one flowchart may be combined with blocks of another flowchart. It will be understood that each block in the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, may be implemented by machine-readable instructions.

[0151] The machine-readable instructions are executed by a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device, for example, to realize the functions described in the description and figures. Specifically, a processor or processing circuit, or a module thereof, executes the machine-readable instructions. Thus, the computer 208, the computer 308, and / or the functional modules of the apparatus 800 (e.g., the receiving module 804 and / or the determining module 806) as well as the functional modules of other devices described herein are implemented by a processor that executes machine-readable instructions stored in a memory, or a processor that operates according to instructions embedded in a logic circuit. The term "processor" should be interpreted broadly to include a CPU, a processing unit, an ASIC, a logic unit, or a programmable gate array, etc. The methods and functional modules are all executed by a single processor or are divided and executed among several processors.

[0152] Such machine-readable instructions may also be stored in computer-readable storage capable of directing a computer or other programmable data processing device to operate in a particular mode.

[0153] Such machine-readable instructions may also be loaded into a computer or other programmable data processing device such that the computer or other programmable data processing device performs a sequence of operations to create a computer-implemented process, such that the instructions executing on the computer or other programmable device implement the functions specified by the blocks in the flowcharts and / or block diagrams.

[0154] Furthermore, the teachings herein may be implemented in the form of a computer program product, the computer program product being stored on a storage medium and comprising a plurality of instructions for causing a computing device to perform the methods according to the embodiments of the present disclosure.

[0155] An element or step described in relation to one embodiment may be combined with or substituted by an element or step described in relation to another embodiment. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the term "comprises" does not exclude other elements or steps, and the singular does not exclude the plural. A single processor or other unit may fulfill several items recited in the claims. The mere fact that certain means are recited in mutually different dependent claims does not indicate that a combination of these means cannot be used to advantage. A computer program may be stored or distributed on a suitable medium, such as an optical storage medium or a solid-state medium, supplied together with or as part of other hardware, or distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems. Any reference signs in the claims should not be interpreted as limiting the scope.

Claims

1. Receiving first imaging data and second imaging data obtained by an imaging system of a subject illuminated by first illumination in a first spectral band and second illumination in a second spectral band having a spectral content different from that of the first spectral band, wherein the second illumination incident on the subject is polarized, The first imaging data received is such that most of the intensity information in the first imaging data obtained within the first spectral band is derived from the first illumination, and the first color filter of the imaging system allows at least a part of the first spectral band to enter the imaging system and prevents at least a part of the second spectral band from entering the imaging system. As a result, it is obtained within the first spectral band, The second imaging data received is such that most of the intensity information in the second imaging data obtained within the second spectral band is derived from the second illumination, and the second color filter of the imaging system allows at least a part of the second spectral band to enter the imaging system and prevents at least a part of the first spectral band from entering the imaging system. As a result, it is obtained within the second spectral band, The first illumination and the second illumination are incident on the imaging system through the imaging system polarizer of the imaging system according to the polarization states of the reflected first illumination and the second illumination received by the imaging system after reflection from the skin surface of the subject, so that the specularly reflected and diffusely reflected first illumination enters the imaging system and the diffusely reflected second illumination enters the imaging system, step; Determining information regarding specular reflection from the skin surface of the subject by comparing the first imaging data and the second imaging data and having, The subject is a human, computer-implemented method.

2. The computer-implemented method according to claim 1, wherein a time frame in which the first imaging data is acquired at least partially overlaps with a time frame in which the second imaging data is acquired.

3. The computer-implemented method according to claim 2, wherein the first imaging data and the second imaging data are acquired simultaneously.

4. The computer-implemented method according to any one of claims 1 to 3, comprising causing the imaging system to receive the first illumination and the second illumination that are reflected and have an electric field component parallel to the polarization axis of the imaging system polarizer, and attenuating the first illumination and the second illumination that are reflected and have an electric field component perpendicular to the polarization axis.

5. The computer-implemented method according to any one of claims 1 to 4, wherein the information regarding specular reflection indicates the gloss level of the subject's skin, and the gloss level is determined by calculating a difference between the intensity information in the first imaging data and the intensity information in the second imaging data.

6. The first color filter and the second color filter are part of a color filter array that enables at least one imaging device of the imaging system to acquire the first imaging data within the first spectral band and the second imaging data within the second spectral band, and the computer-implemented method further comprises extracting the first imaging data separately from the second imaging data from the raw imaging data acquired by the at least one imaging device. The computer-implemented method according to any one of claims 1 to 5.

7. A tangible machine-readable medium comprising instructions that, when executed by a processing circuit, cause the processing circuit to perform the computer-implemented method according to any one of claims 1 to 6.

8. An apparatus comprising a processing circuit, wherein the processing circuit is A receiving module that receives first imaging data and second imaging data acquired by an imaging system of a subject illuminated by first illumination in a first spectral band and second illumination in a second spectral band including spectral content different from the first spectral band, wherein the second illumination incident on the subject is polarized, The first imaging data received is such that most of the intensity information in the first imaging data acquired within the first spectral band is derived from the first illumination, the first color filter of the imaging system allows at least a part of the first spectral band to enter the imaging system, and as a result of preventing at least a part of the second spectral band from entering the imaging system, it is acquired within the first spectral band, The second imaging data received is such that most of the intensity information in the second imaging data acquired within the second spectral band is derived from the second illumination, the second color filter of the imaging system allows at least a part of the second spectral band to enter the imaging system, and as a result of preventing at least a part of the first spectral band from entering the imaging system, it is acquired within the second spectral band, A receiving module in which the first illumination and the second illumination enter the imaging system through an imaging system polarizer of the imaging system according to the polarization states of the reflected first illumination and the second illumination received by the imaging system after reflection from the skin surface of the subject so that the specularly reflected and diffusely reflected first illumination enters the imaging system and the diffusely reflected second illumination enters the imaging system, A determination module that determines information regarding specular reflection from the skin surface of the subject by comparing the first imaging data and the second imaging data, The subject is a human, the device.

9. The apparatus according to claim 8, wherein the determination module determines a measurement value of the glossiness of the skin of the subject based on a comparison between the first imaging data and the second imaging data.

10. The apparatus according to claim 8 or 9, further comprising the imaging system and / or a lighting system that provides the first lighting and the second lighting.

11. The imaging system polarizer causes the first lighting and the second lighting that are reflected and have an electric field component parallel to the polarization axis of the imaging system polarizer to enter the imaging system, and prevents the first lighting and the second lighting that are reflected and have an electric field component perpendicular to the polarization axis from entering the imaging system. The apparatus according to claim 10.

12. The lighting system includes a lighting system polarizer that polarizes the second lighting directed at the subject, and the polarization axis of the imaging system polarizer is orthogonal to the polarization axis of the lighting system polarizer. The apparatus according to claim 11.

13. The lighting system is configured such that the first lighting directed at the subject is unpolarized, or the lighting system includes an additional lighting system polarizer that polarizes the first lighting directed at the subject such that the polarization state of the first lighting directed at the subject is orthogonal to the polarization state of the second lighting directed at the subject. The apparatus according to any one of claims 10 to 12.

14. The lighting system is configured to direct the first lighting and the second lighting at the subject such that both the specular reflection component and the diffuse reflection component of the first lighting and the second lighting reflected from the skin surface of the subject are directed into the imaging system for internal light entry according to the polarization states of the reflected first and second lighting. The apparatus according to any one of claims 10 to 13.

15. The imaging system includes at least one imaging device and an optical filter array, the optical filter array includes the first color filter and the second color filter, and the optical filter array Pass at least a portion of the first spectral band into the imaging system through a set of first pixels of the at least one imaging device such that most of the intensity information in the first imaging data acquired within the first spectral band is derived from the first illumination; Pass at least a portion of the second spectral band into the imaging system through a set of second different pixels of the at least one imaging device such that most of the intensity information in the second imaging data acquired within the second spectral band is derived from the second illumination The apparatus according to any one of claims 10 to 14, which performs the above.

Images