Signal-to-noise ratio targeting

Abstract

In an embodiment, a method (100) is described. The method comprises receiving (102) data of an ambient corrected image corresponding to an object illuminated by an illumination unit (206), the illumination unit providing time-modulated illumination having a modulation frequency higher than a frame acquisition rate used by an imaging device (204) for obtaining a set of images, the set of images having a different spatial intensity modulation pattern in each image. The ambient corrected image is constructed from the obtained set of images. The method (100) further comprises determining (104) a signal-to-noise ratio, SNR, of at least a portion of the ambient corrected image. In response to determining that the SNR is lower than a target SNR, the method causes (106) an indication of an illumination parameter to be sent to the illumination unit to increase a magnitude modulation depth of the time-modulated illumination used for illuminating the object when the imaging device acquires a subsequent set of images, the subsequent set of images used for constructing a subsequent ambient corrected image.

Description

Technical Field

[0001] The present invention relates to methods, apparatus, and tangible machine-readable media for imaging under certain conditions. Background Technology

[0002] A topic of interest in the field of non-invasive measurement and monitoring involves skin sensing for personal care and health applications. Skin sensing systems are being developed that promise to quantify the skin and monitor its features, providing users with information that is too small to be detected, too weak to be noticed, or too slow to be followed. To deliver user-acceptable results, such skin sensing systems may need to provide sensitivity and specificity when performing skin sensing. If measurements taken by such skin sensing systems prove robust and reliable, users can build trust in these systems.

[0003] Imaging-based skin sensing systems may need to determine information that can be affected by parameters that are difficult to control, such as variations in ambient light. For example, certain uncontrolled environments (such as in a user's home) may be characterized by ambiguous and / or potentially variable ambient light. Such uncontrolled environments may lead to inaccurate measurements of the user's skin, which in turn may result in unacceptable or untrustworthy results for the user. The imaging performance of some cameras used in some imaging-based skin sensing systems (such as smartphone cameras) may be variable, making the imaging data unpredictable or unreliable.

[0004] Imaging-based skin sensing systems can implement various image processing techniques to determine certain information about the user's skin and / or ambient lighting conditions. Certain ambient lighting conditions can affect the results obtained using some image processing techniques, potentially leading to impaired or inconsistent measurements of the user's skin.

[0005] Note that patent application WO 2012 / 175703 A1 discloses a method for generating control signals based on an observer's movement or posture. According to this patent application, a display device generates light of varying intensity to illuminate an object. Multiple images with a constant amount of ambient light are captured, the display device generates light, and the amount of light displayed varies between the images. Summary of the Invention

[0006] The aspects or embodiments described herein relate to improving imaging under certain conditions. The aspects or embodiments described herein can eliminate one or more problems associated with certain image processing techniques that may be affected by certain ambient lighting conditions.

[0007] In a first aspect, a method is described. This method is computer-implemented. The method includes: receiving data corresponding to an environment-corrected image of an object illuminated by an illumination unit. The illumination unit provides time-modulated illumination. The time-modulated illumination has a modulation frequency higher than the frame acquisition rate used by an imaging device to acquire an image set. Due to the time-modulated illumination and the imaging device operating in a rolling shutter mode, the image set has different spatial intensity modulation patterns in each image. Information from the acquired image set is used to construct an environment-corrected image. The effect of ambient illumination is reduced in the environment-corrected image. The method further includes: determining the signal-to-noise ratio (SNR) of at least a portion of the environment-corrected image. In response to determining that the SNR is lower than a target SNR, the method further includes: causing an indication of illumination parameters to be sent to the illumination unit to increase the amplitude modulation depth of the time-modulated illumination used to illuminate the object when the imaging device acquires a subsequent image set. The subsequent image set is used to construct a subsequent environment-corrected image with an increased SNR for at least a portion of the subsequent environment-corrected image.

[0008] In a second aspect, a method is described. This method is computer-implemented. The method includes: receiving data corresponding to an environmentally corrected image of an object illuminated by time-modulated illumination. The environmentally corrected image reduces the effect of ambient illumination caused by a light source. The method further includes: determining the signal-to-noise ratio (SNR) of at least a portion of the environmentally corrected image. If the determined SNR is lower than a target SNR, the method further includes: providing indications of illumination parameters to achieve the target SNR for at least that portion of the image.

[0009] The following describes certain embodiments relating to the first and / or second aspects.

[0010] In some embodiments, the target SNR is based on the object's skin type.

[0011] In some embodiments, a target SNR is associated with a factor A, for which various different skin types are associated with obtaining an image with at least a target SNR. Factor A is defined as the ratio of the ambient light level incident on the object to the total amount of light incident on the object. The total amount of light refers to the combination of the ambient light level and the modulation depth of the amplitude modulation provided by the illumination unit. This indicator is configured to vary the modulation depth depending on the skin type to obtain the same factor A for each of the different skin types.

[0012] In some embodiments, the indication is configured to control the ambient light level of the object to increase the SNR for at least that portion of a subsequent ambient-corrected image.

[0013] In some embodiments, the instruction is configured to cause a reduction in the ambient light level to increase the SNR for at least that portion of the ambient-corrected image.

[0014] In some embodiments, the indication of lighting parameters includes control instructions for specifying the level of ambient light to be provided by a source of ambient light.

[0015] In some embodiments, the method includes using the indication to control the power supply to the source such that a specified ambient light level is provided.

[0016] In some embodiments, the indication of lighting parameters includes instructions to indicate the need for a specified ambient light level and / or the need for user action to provide a specified ambient light level.

[0017] In some embodiments, the method further includes causing the imaging device to acquire an image set of the object while the object is illuminated by time-modulated illumination. The method may further include determining an environment-corrected image based on the image set of the object.

[0018] In some embodiments, the method further includes: causing the imaging device to acquire a subsequent set of images of the object while the object is illuminated by time-modulated illumination having an increased amplitude modulation depth. The method may further include: determining a subsequent environment-corrected image based on the subsequent set of images of the object.

[0019] In some embodiments, before receiving data corresponding to an environmentally corrected image, the method includes: selecting an initial amplitude modulation depth of time-modulated illumination and / or an initial ambient light level based on the skin type of the object.

[0020] In a third aspect, a tangible machine-readable medium is described. This tangible machine-readable medium stores instructions that, when executed by at least one processor, cause the at least one processor to perform the method according to any of the foregoing aspects or embodiments.

[0021] In a fourth aspect, an apparatus is described. The apparatus includes a processing circuit system. This processing circuit system includes a receiving module, a determining module, and an indicating module. The receiving module is configured to receive data corresponding to an environment-corrected image of an object illuminated by an illumination unit that provides time-modulated illumination having a modulation frequency higher than the frame acquisition rate used by an imaging device to acquire an image set. Due to the time-modulated illumination and the imaging device operating in a rolling shutter mode, the image set has different spatial intensity modulation patterns in each image. Information from the acquired image set is used to construct an environment-corrected image. The effect of ambient light is reduced in the environment-corrected image. The determining module is configured to determine the signal-to-noise ratio (SNR) of at least a portion of the environment-corrected image. The indicating module is configured to, in response to determining that the SNR is lower than a target SNR, send an indication of illumination parameters to the illumination unit to increase the amplitude modulation depth of the time-modulated illumination used to illuminate the object when the imaging device acquires a subsequent image set. The subsequent image set is used to construct a subsequent environment-corrected image.

[0022] In a fifth aspect, an apparatus is described. The apparatus includes a processing circuitry system. The processing circuitry system includes a receiving module for receiving data corresponding to an environmentally corrected image of an object illuminated by time-modulated illumination. The environmentally corrected image reduces the effect of ambient light caused by the light source. The processing circuitry system further includes a determining module for determining the signal-to-noise ratio (SNR) of at least a portion of the environmentally corrected image. The apparatus further includes an indicating module for providing an indication of illumination parameters to achieve the target SNR for at least that portion of the image if the determined SNR is lower than a target SNR.

[0023] The following describes certain embodiments relating to the fourth and / or fifth aspects.

[0024] In some embodiments, the device further includes a control module for controlling: an illumination unit for providing time-modulated illumination; and / or a source of ambient light.

[0025] In some embodiments, the device further includes a user interface for providing instructions based on indications of lighting parameters to indicate the need for a specified ambient light level and / or the need for user action to provide a specified ambient light level.

[0026] These and other aspects of the invention will be apparent from the embodiments described below and will be illustrated with reference to these embodiments. Attached Figure Description

[0027] Exemplary embodiments of the invention will now be described by way of example only with reference to the following accompanying drawings, in which:

[0028] Figure 1 This refers to an improved imaging method under certain conditions, according to embodiments;

[0029] Figure 2 This is a schematic diagram of a system for improving imaging under certain conditions, according to an embodiment;

[0030] Figure 3a -b is a schematic diagram of a system for improving imaging under certain conditions according to an embodiment;

[0031] Figure 4 Diagrams depicting implementations of certain methods described herein according to embodiments are shown;

[0032] Figure 5 This refers to an improved imaging method under certain conditions, according to embodiments;

[0033] Figure 6 This is a schematic diagram of a machine-readable medium for improving imaging under certain conditions, according to an embodiment;

[0034] Figure 7 This is a schematic diagram of an apparatus for improving imaging under certain conditions, according to an embodiment; and

[0035] Figure 8 This is a schematic diagram of an apparatus for improving imaging under certain conditions, according to an embodiment. Detailed Implementation

[0036] Figure 1 A method 100 for improving imaging under certain conditions is illustrated (e.g., a computer-implemented method). Method 100 may be implemented by a computer (such as a user device) or a server or cloud-based service (e.g., communicatively connected to a user device). Examples of user devices include smart devices, such as smartphones, tablets, smart mirrors, or any other device capable of displaying an image of an object or a representation of an image.

[0037] Method 100 includes, at block 102, receiving data corresponding to an environment-corrected image of an object illuminated by an illumination unit providing time-modulated illumination having a modulation frequency higher than the frame acquisition rate used by an imaging device to acquire an image set. Due to the time-modulated illumination and the imaging device operating in a rolling shutter mode, the acquired image set has different spatial intensity modulation modes in each image. Information from the acquired image set is used to construct the environment-corrected image. The effect of ambient light is reduced in the environment-corrected image. The environment-corrected image can be obtained by acquiring an image set of the object and implementing a technique that uses a spatial intensity modulation mode in each image to reduce the effect of ambient light caused by a light source (or multiple light sources) (such as indoor lighting or sunlight).

[0038] Block 102 refers to an example technique for reducing the effect of ambient light caused by a light source, which includes acquiring an image set (e.g., at least three images) of an object while temporally modulating (e.g., via periodic temporal modulation) the intensity of illumination provided by an illumination unit for illuminating the object. The imaging device for acquiring the image set operates in a rolling shutter mode (or another mode involving acquiring different spatial portions of the image at different times within a frame) at a frame acquisition rate lower than the modulation frequency of the temporally modulated illumination (e.g., in one example, for a frame rate of 30 frames per second (fps), the modulation frequency may be at least 70 Hz). In other similar terms, the modulation frequency is higher than the frame acquisition rate used to acquire the image set to construct an environment-corrected image. This means that as the imaging device acquires each image, the object is illuminated at varying intensity levels as the imaging device performs its scan, such that some pixels of the imaging device record a higher intensity level than other pixels of the imaging device. As a result, each consecutive image in the image set may have a different spatial intensity modulation pattern. Information from this image set can be used to reconstruct the environment-corrected image if ambient light is removed, normalized, or at least reduced. In other words, in an ambient-corrected image, an object may appear to be uniformly illuminated, or the ambient lighting may not change significantly across the object's skin surface.

[0039] As mentioned above, in block 102, method 100 refers to receiving data corresponding to an ambient light corrected image. This data may refer to data stored in memory (such as the memory of the computer implementing method 100 or another memory) or data obtained after implementing techniques for reducing the effect of ambient light on the image. In other similar terms, the computer implementing method 100 may access the memory storing the data and / or may itself implement techniques for reducing the effect of ambient light on the image.

[0040] Method 100 includes: at block 104, determining the signal-to-noise ratio (SNR) of at least a portion of the environmentally corrected image.

[0041] This portion of the environment-corrected image can refer to any area of ​​the image. In some embodiments, multiple portions of the environment-corrected image can be selected, and the SNR of those multiple portions can be determined.

[0042] In some embodiments, SNR may be defined by the following formula:

[0043]

[0044] This allows the average intensity of pixel values ​​within each portion, along with the standard deviation of those pixel values' intensity, to be determined. The location, size, and / or shape of the selected region (portion) may affect the SNR calculation. Therefore, in some embodiments, the selection of the (multiple) locations, sizes, and / or shapes of the portions from which the SNR is determined may be based on factors that take into account variations across the image. For example, some portions of an image may have different SNRs than other portions of the image (e.g., due to factors such as the reflectivity of skin and / or hair surfaces, background information, and / or other features within the image). For the purposes of the methods described below, determining the SNR of the entire image may or may not provide meaningful information. Determining the SNR of at least one portion of the image may provide more useful information and / or allow for taking into account variations in SNR across the image. However, in some embodiments, the SNR of the entire image may be determined.

[0045] If the determined SNR is lower than the target SNR, method 100 includes, at block 106, in response to determining that the SNR is lower than the target SNR, causing an indication of illumination parameters to be sent to the illumination unit to increase the amplitude modulation depth of the temporally modulated illumination of the object being illuminated when the imaging apparatus acquires a subsequent image set. The subsequent image set is used to construct subsequent environmentally corrected images such that at least a portion of the subsequent environmentally corrected images (e.g., corresponding to the portion of the environmentally corrected image in which the SNR was determined) has an increased SNR. The target SNR may refer to an SNR that allows for improved imaging under certain conditions, such as when ambient lighting conditions are not optimal or unsuitable for achieving consistent and / or reliable results when performing skin sensing measurements or analyses based on environmentally corrected images. For example, the target SNR may refer to a threshold (e.g., a predetermined threshold) in which acceptable image quality is obtained for performing consistent and / or reliable skin sensing measurements. Different target SNRs may be defined depending on the type of skin sensing measurement or analysis. For example, some skin sensing measurements or analyses may require a higher SNR than other types of skin sensing measurements in order to produce consistent and / or reliable results.

[0046] As will be described in more detail, in some embodiments, the target SNR may vary depending on various factors, such as the skin type in the analysis. Therefore, the target SNR can be selected based on various factors that may affect skin sensing measurements or analyses performed using environmentally corrected images.

[0047] By appropriately selecting the target SNR, Method 100 can facilitate improved imaging and / or image processing under certain conditions, resulting in subsequent environmentally corrected images of better quality for certain skin sensing measurements or analysis purposes.

[0048] Furthermore, imaging can be improved by providing indications of illumination parameters to increase the SNR for at least that portion of the subsequent environmentally corrected image.

[0049] Increasing the amplitude modulation depth can increase the SNR in subsequent environment-corrected images because the subsequent image set used to construct the subsequent environment-corrected images can each have a spatial modulation mode, wherein in each of the subsequent image sets, the contrast between the maximum and minimum pixel intensity values ​​is increased (due to spatial intensity modulation). By causing an increase in the amplitude modulation depth of the illumination, environment correction techniques using different spatial intensity modulation modes from the subsequent image sets can construct subsequent environment-corrected images with increased SNR (e.g., due to the difference between the maximum and minimum intensities of the dominant ambient light illumination).

[0050] Some embodiments described below refer to indications of lighting parameters that can be provided in various ways to improve SNR.

[0051] In some cases, SNR can be considered to play a significant role in the perceptual quality of environment-corrected images. A sufficiently high SNR is desirable to allow for accurate quantitative measurements from environment-corrected images. To track skin parameters and features over time, it may be useful to output measurements independent of ambient light conditions. Achieving a sufficiently high SNR in environment-corrected images facilitates sufficiently accurate tracking of skin parameters over time.

[0052] Figure 2 An example system 200 for improving imaging, according to certain embodiments, is illustrated. System 200 may implement at least in part some of the methods described herein, such as method 100 above or method 500 below. In some embodiments, certain blocks of system 200 may be omitted.

[0053] System 200 is used by object 202 and includes an imaging system 204 and an illumination unit 206. Imaging system 204 is used to acquire the image mentioned in method 100. Illumination unit 206 is used to provide time-modulated illumination. Imaging system 204 and / or illumination unit 206 may be implemented by a user device (such as the user device described above). Thus, in some embodiments, multiple separate user devices may include imaging system 204 and illumination unit 206, and in other embodiments, the user device may include imaging system 204 and illumination unit 206.

[0054] In some embodiments, the imaging system 204 includes at least one imaging device (e.g., a camera in rolling shutter mode) for capturing images or videos, which is capable of detecting one or more light sources (e.g., sources such as ambient light and / or illumination from the illumination unit 206) that interact with the surface (e.g., skin) of the object 202.

[0055] System 200 further includes computer 208 (e.g., including processing circuitry implemented by a user device, server, or cloud-based service to implement certain methods described herein). Therefore, computer 208 can be communicatively coupled to imaging system 204 and / or illumination unit 206 to send data to and / or receive data from these components. This data can be processed by the processing circuitry of computer 208 and / or stored in memory (e.g., the memory of computer 208 or memory accessible to the processing circuitry of computer 208). In some embodiments, computer 208 can control the operation of imaging system 204 and / or illumination unit 206. In some embodiments, computer 208 includes a controller for controlling illumination parameters (e.g., operating parameters of illumination unit 206) and / or detection parameters (e.g., operating parameters of imaging system 204) and storing and / or processing captured images or videos.

[0056] System 200 includes a light source 210, which in some embodiments may be different from illumination unit 206 and provide ambient light to object 202. However, in some embodiments, illumination unit 206 may provide time-modulated lighting and provide or facilitate ambient light. In some embodiments, system 200 may not include light source 210. In some embodiments, light source 210 may refer to a smart lamp (e.g., a smart light-emitting diode (LED) or other light source whose light output can be controlled by a computer (such as computer 208)). For example, some smart lamps may be controlled by data provided via a wireless or wired communication link, for example, to control the intensity level and / or color output of the smart lamp. In this embodiment, computer 208 is communicatively coupled to light source 210 (e.g., to allow computer 208 to control the output of light source 210). In some embodiments, computer 208 may not be able to control the output of light source 210. Light source 210 may be the sole source of ambient light or may facilitate ambient light from another source (such as the sun).

[0057] Figure 3a -b indicates the methods used to implement some of the methods described in this article (such as, Figure 1 The method 100) of the system 300. The system 300 includes a system corresponding to Figure 2 Some features of system 200, and the reference numerals of these features are consistent with... Figure 2 Those were increased by 100 compared to the others.

[0058] The imaging device 304 of system 300 images the object 302. Computer 308 records the average value and standard deviation of the signal acquired by imaging device 304 based on grayscale values ​​or the number of photons recorded within an integration time. The SNR is calculated using the example formula described above. In some embodiments, the SNR is calculated independently of the object's skin type and ambient lighting conditions.

[0059] Figure 3a An environmentally corrected image 312 of object 302 is shown, wherein the SNR of at least one portion 314 of image 312 is determined. (As shown in...) Figure 3a As can be seen, the illumination unit 306 provides a certain level of modulated illumination (e.g., modulation depth described below). However, it is obvious that in Figure 3a In this case, the image 312 is of poor quality, causing at least one portion 314 of the image 312 to have a poor SNR (e.g., it is 'blurry' or noisy).

[0060] According to some of the methods described herein, computer 308 determines the SNR of image 312 in [the specified range]. Figure 3a The SNR is not high enough (i.e., it is below the target SNR), and therefore, in situations such as... Figure 3b In some of the depicted embodiments, computer 308 causes lighting unit 306 to provide different modulated lighting levels (e.g., more than...). Figure 3a Greater modulation depth—this is achieved through Figure 3b The shadow ratio adjacent to the lighting unit 306 Figure 3a (Depicted in darker tones). Figure 3b Image 312 in the image has a higher density than Figure 3a The image 312 achieves higher quality (and therefore higher SNR). After at least one iteration of method 100, at least one portion 314 of image 312 can reach the target SNR.

[0061] In some embodiments, analysis of the SNR of each environmental correction image 312 may result in the generation of feedback signals for controlling the lighting unit 306 (e.g., to iteratively achieve the target SNR).

[0062] Factor 'A' can be defined as the ratio of ambient light ('ambient [lux]') to the total amount of light falling on the skin surface of an object (i.e., ('illuminance + ambient [lux]')).

[0063]

[0064] 'Ambient' refers to light from a light source (such as a smart lamp or any other lamp in the object room and / or sunlight), and 'illuminance' refers to the amplitude or modulation depth of the time-modulated lighting provided by the lighting unit 306.

[0065] When the ambient light level is zero or much less than the amplitude of time-modulated lighting, the value 'A' may be low (e.g., A = 0). This can occur when the object is in a dark room, without sunlight, etc.

[0066] When the ambient light level is high or much greater than the amplitude of time-modulated lighting, the value 'A' may be high (e.g., A = 0.9). This may occur when the object is in a bright room, under bright sunlight, etc.

[0067] In other words, the value of factor 'A' depends on the relative value of the ambient light level and the amplitude of the time-modulated lighting.

[0068] In some embodiments, an A value is recorded for each captured image, such that color information can be normalized to a specified A value in order to track skin features over time.

[0069] Figure 4 Includes a chart (a scatter plot of SNR (in dB) versus A) that shows the correlation between a fixed factor 'A' and obtaining a sufficient SNR in five example images of the object (associated with the following 'A' values: 0, 0.25, 0.5, 0.75, and 0.9).

[0070] The blocks embedded in each image in the table represent portions of the image sampled for different skin types (labeled as skin type blocks 1 to 6 in the chart's legend). The chart also shows the results with SNR contrast A for the six skin types.

[0071] As can be seen, image quality significantly decreases with increasing 'A'. For example, regardless of skin type, the SNR is relatively high for A=0, while for all skin types, the SNR is relatively low for A=0.9 (and lower than the target SNR in the chart). Therefore, based on the understanding of how 'A' varies with the relative value of ambient light level and the amplitude of time-modulated illumination, low ambient light levels can be associated with high SNR, and high ambient light levels can be associated with low SNR.

[0072] As will be apparent from the image for A=0, each skin type is associated with a sufficiently high SNR (e.g., greater than the target SNR). This is depicted by a graph showing how the SNR for each skin type is higher than the target SNR (represented by a horizontal dashed line at 14 dB). However, for increasingly larger values ​​of A (associated with increasing ambient light levels), the SNR can decrease to below the target SNR, depending on the subject's skin type.

[0073] For example, some skin types (e.g., skin types labeled >4) require a factor 'A' smaller than that of some other skin types (e.g., skin types labeled ≤4) in order to achieve the target SNR (i.e., 14 dB in this case). In other words, for all skin types, the 'A' value may need to be much lower (e.g., 0.1) to achieve the target SNR, while for a subset of skin types (e.g., skin types labeled <4), the 'A' value can be higher (e.g., 0.6).

[0074] Therefore, as factor 'A' increases, image quality may degrade, potentially impairing skin measurements and analysis in high-light environments. This can lead to inconsistent results over time (e.g., due to variations in ambient light levels on different days). The degradation can be non-linear for different skin types, and therefore 'A' may need to be fixed so that all targeted skin types achieve an SNR higher than the design value. For example, 'A' could be set to a low value (e.g., A = 0.1) so that the SNR is suitable for all skin types and the performance of skin sensing measurements and analysis is acceptable. In other similar terms, factor A can be associated with SNR based on skin type, such that there exists a factor A that provides acceptable SNR results for all skin types.

[0075] As previously mentioned, the instructions of method 100 may lead to improvements in SNR (and therefore, improvements in skin measurements and analysis) in various ways. To increase SNR or achieve a target SNR (which may be associated with a value of factor 'A'), the ambient light level and / or the amplitude of time-modulated lighting may be altered to achieve the specified value of factor A. For example, if the SNR is below the target SNR, the ambient light level may be reduced and / or the amplitude of time-modulated lighting may be increased.

[0076] Therefore, in some embodiments, the target SNR is based on the object's skin type.

[0077] In some embodiments, a target SNR is associated with a factor A, for which various different skin types are associated with obtaining an image with at least a target SNR. Factor A can be defined as the ratio of the ambient light level incident on the object to the total amount of light incident on the object. The total amount of light can refer to a combination of the ambient light level and the modulation depth of the amplitude modulation provided by the illumination unit. This indication can be configured to vary the modulation depth depending on the skin type to obtain the same factor A for each of the different skin types.

[0078] In some embodiments, the indication is configured to control the ambient light level of the object to increase the SNR and / or achieve a target SNR for at least that portion of a subsequent ambient-corrected image.

[0079] In some embodiments, the indication is configured to cause a reduction in the ambient light level to increase the SNR and / or achieve a target SNR for at least that portion of a subsequent ambient-corrected image. Amplitude modulation, or 'modulation depth,' may refer to the difference between the maximum and minimum amplitudes of time-modulated lighting. In some cases, the ambient light level may be considered 'DC,' a constant or slowly varying light level, while time-modulated lighting may be considered 'AC,' or a periodically varying light level.

[0080] With an increased amplitude modulation depth, the indication of lighting parameters can include different amplitude modulation depths to be provided by the lighting units used to provide time-modulated lighting. For example, if the ambient light level is too high, making the value 'A' too high to achieve the target SNR, the amplitude modulation depth can be increased. For example, the difference between the maximum and minimum intensity provided by lighting unit 206 can be increased. In some cases, if the ambient light level is low, the amplitude modulation depth can be appropriately reduced, for example, to save energy.

[0081] In some embodiments, the indication of lighting parameters includes control instructions for specifying the level of ambient light to be provided by a source of ambient light. For example, as by Figure 2 As shown, computer 208 is communicatively connected to light source 210. Therefore, in some embodiments, if the current SNR in the environmental correction image is lower than the target SNR, computer 208 may cause light source 210 to provide less light, thereby reducing the value of 'A' (and thus increasing the SNR to increase the SNR and / or achieve the target SNR).

[0082] In some embodiments, the indication of lighting parameters includes instructions (e.g., user instructions) to indicate (e.g., to the object itself or an individual / user performing measurements on the object's skin using some of the methods described herein) that a specified ambient light level is required and / or an action (e.g., an action performed by the object and / or the user) is needed to provide or achieve the specified ambient light level. For example, if the ambient light is too high (making A too high), the indication may include an instruction to notify the user that they need to take an action to reduce the ambient light level. For example, the instruction may suggest an action to the object or user that prompts the object to move to a darker room or reduce the ambient light level in some other way, such as turning off the lights or blocking light from entering the room through a window. The instruction may be provided on the user device or user interface in the form of a prompt (e.g., a visual cue on the device screen and / or an audible cue from the device's audio output).

[0083] Any combination of the strategies described above can be used to control factor 'A' in order to achieve the target SNR.

[0084] Figure 5 A method 500 for improving imaging under certain conditions is illustrated. Method 500 includes blocks 102 to 106 of method 100 and, in some embodiments, may be implemented by a computer 208. In some embodiments, certain blocks of method 500 may be omitted, and / or may be used in conjunction with… Figure 5 The blocks of method 500 are implemented in different sequences as shown.

[0085] In some embodiments, prior to receiving data corresponding to an environmentally corrected image (i.e., at block 102), at block 502, the initial amplitude modulation depth of the time-modulated illumination and / or the initial ambient light level are selected based on the object's skin type. For example, the object's skin type can be an initial input used in method 500 to reduce the number of iterations required to increase the SNR and / or achieve a target SNR (e.g., in some cases, multiple iterations to increase the SNR may be necessary to achieve a target SNR). If the system knows the object's skin type before implementing method 100, the initial amplitude modulation depth can be appropriately selected to avoid unnecessary iterations of some of the methods described herein. For example, if the user's skin type is known to be '6' (see...), the initial amplitude modulation depth can be appropriately selected to avoid unnecessary iterations of some of the methods described herein. Figure 4 If less ambient light and / or greater amplitude modulation depth are required, this information can be indicated to the object / user, lighting unit 206 and / or light source 210, so that method 500 can achieve the target SNR faster in the ambient correction image.

[0086] As previously mentioned, the environmental correction image is determined based on the spatial intensity modulation pattern in each of a set (e.g., consecutive) images of the object.

[0087] Therefore, in some embodiments, method 500 includes, at block 504, causing an imaging device (e.g., the imaging device of imaging system 204) to acquire an image set of the object while the object is illuminated by time-modulated illumination. Block 504 further includes determining an environment-corrected image based on the image set of the object.

[0088] In some embodiments, method 500 includes, at block 506, using the indication to cause lighting unit 206 to provide time-modulated lighting with different amplitude modulation depths. For example, the indication may include a signal configured to cause lighting unit 206 (e.g., a light-emitting diode or other lighting unit implemented by a user device or other suitable means) to change its lighting output according to the signal. For example, electrical parameters (such as the current and / or voltage supplied to lighting unit 206) may vary over time to provide time-modulated lighting.

[0089] In some embodiments, method 500 includes, at block 508, causing the imaging device to acquire a subsequent set of images of the object while the object is illuminated by time-modulated illumination having an increased amplitude modulation depth (e.g., according to block 506). Block 508 further includes: determining a subsequent environment-corrected image based on the subsequent set of images of the object.

[0090] In some embodiments, method 500 includes, at block 510, using the indication to control the power supply for the source (e.g., light source 210) such that a specified ambient light level is provided. For example, the power supply for light source 210 may be configured to provide energy to allow light source 210 to generate ambient light, and the indication may control the amount of power (e.g., current and / or voltage) supplied to light source 210 to change the light output from light source 210.

[0091] Figure 6 A tangible machine-readable medium 600 is shown storing instructions 602 that, when executed by at least one processor 604, cause the at least one processor to perform certain methods described herein, such as method 100 or method 500.

[0092] In this embodiment, instruction 602 includes instruction 606 for causing the at least one processor 604 to implement block 102 of method 100. Instruction 602 further includes instruction 608 for causing the at least one processor 604 to implement block 104 of method 100. Instruction 602 further includes instruction 610 for causing the at least one processor 604 to implement block 106 of method 100.

[0093] Figure 7An apparatus 700 is shown that can be used to implement certain methods described herein (such as method 100 and / or method 500). The apparatus 700 may include having corresponding... Figure 2 The device 700 includes modules that provide certain functionalities of the system 200 (such as its computer 208). The device 700 includes a processing circuitry system 702.

[0094] The processing circuit system 702 includes a receiving module 704. The receiving module 704 is configured to receive data corresponding to an environment-corrected image of an object illuminated by an illumination unit (e.g., illumination unit 206), which provides time-modulated illumination having a modulation frequency higher than the frame acquisition rate used by the imaging device (e.g., imaging device 204). The imaging device acquires an image set, each image having a different spatial intensity modulation pattern due to the time-modulated illumination and the imaging device operating in a rolling shutter mode. Information from the acquired image set is used to construct the environment-corrected image. The effect of ambient light is reduced in the environment-corrected image.

[0095] The processing circuitry system 702 further includes a determination module 706. The determination module 706 is configured to determine the signal-to-noise ratio (SNR) of at least a portion of the environmentally corrected image.

[0096] The processing circuit system 702 further includes an indication module 708. The indication module 708 is configured to, in response to determining that the SNR is lower than a target SNR, send an indication of the illumination parameters to the illumination unit to increase the amplitude modulation depth of the temporally modulated illumination of the illuminated object when the imaging device acquires a subsequent image set. The subsequent image set is used to construct subsequent environmentally corrected images such that at least a portion of the subsequent environmentally corrected images has the increased SNR.

[0097] Figure 8 An apparatus 800 is shown that can be used to implement certain methods described herein (such as method 100 and / or method 500). The apparatus 800 includes a processing circuitry system 802, which includes… Figure 7 The processing circuit system 702.

[0098] In some embodiments, the device 800 further includes a control module 804 for controlling: an illumination unit 206 for providing time-modulated illumination; and / or a source of ambient light (e.g., a light source 210).

[0099] In some embodiments, device 800 further includes a user interface 806 for providing instructions based on indications of lighting parameters to indicate the need for a specified ambient light level and / or the need for user action to provide or achieve a specified ambient light level. In this embodiment, user interface 806 is part of device 800 (e.g., if device 800 includes a user device and user interface 806). In some other embodiments, user interface 806 may be a separate entity (e.g., a means for implementing certain methods described herein, separate from computer 208).

[0100] In some cases, any of the modules described above (e.g., receiving module 704, determining module 706, indicating module 708 and / or control module 804) may include at least one dedicated processor (e.g., application-specific integrated circuit (ASIC) and / or field-programmable gate array (FPGA) etc.) for implementing the functions of the module.

[0101] In some cases, the modules described above (e.g., receiving module 704, determining module 706, indicating module 708, and / or controlling module 804) may include at least one processor for implementing instructions that cause the at least one processor to perform the functions of the modules described above. In such examples, these instructions may be stored in a machine-readable medium (not shown) accessible to the at least one processor. In some examples, the module itself includes a machine-readable medium. In some examples, the machine-readable medium may be separable from the module itself (e.g., the at least one processor of the module may be provided to communicate with the machine-readable medium to access the instructions stored therein).

[0102] Although the invention has been illustrated and described in detail in the accompanying drawings and the foregoing description, such illustrations and descriptions are to be considered illustrative or exemplary rather than limiting; the invention is not limited to the disclosed embodiments.

[0103] One or more features described in one embodiment may be combined with or replace features described in another embodiment. For example, features may be based on... Figure 2 Or modify the features described in systems 200, 300, machine-readable medium 600, and / or devices 700, 800 of Figure 3. Figure 1 or Figure 5 Methods 100 and 500, and vice versa.

[0104] The embodiments in this disclosure may be provided as a method, a system, or as a combination of machine-readable instructions and processing circuitry. Such machine-readable instructions may be included on a non-transitory machine (e.g., computer) readable storage medium (including, but not limited to, disk storage devices, CD-ROMs, optical storage devices, etc.) having computer-readable program code thereon or thereon.

[0105] This disclosure is described with reference to flowchart illustrations and block diagrams of methods, apparatus, and systems according to embodiments of this disclosure. Although the flowcharts described above illustrate a particular execution order, the execution order may differ from the described execution order. A block described in one flowchart may be combined with a block in another flowchart. It should be understood that each block in a flowchart and / or block diagram, and combinations of blocks in flowcharts and / or block diagrams, can be implemented by machine-readable instructions.

[0106] Machine-readable instructions can be executed, for example, by a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device, to implement the functions described in the specification and drawings. In particular, a processor or processing circuitry system, or a module thereof, can execute machine-readable instructions. Therefore, functional modules of system 200 and / or devices 700, 800 (e.g., receiving module 704, determining module 706, indicating module 708, and / or control module 804), as well as other means described herein, can be implemented by a processor that executes machine-readable instructions stored in memory or by a processor that operates according to instructions embedded in a logic circuitry system. The term 'processor' will be interpreted broadly to include CPUs, processing units, ASICs, logic units, or programmable gate arrays, etc. These methods and functional modules can be executed entirely by a single processor or divided among several processors.

[0107] Such machine-readable instructions can also be stored in a computer-readable storage device that can instruct a computer or other programmable data processing device to operate in a specific mode.

[0108] Such machine-readable instructions may also be loaded onto a computer or other programmable data processing device to cause the computer or other programmable data processing device to perform a series of operations to produce computer-implemented processing. Thus, the instructions that execute on the computer or other programmable device implement the functions specified by the blocks(s) in the flowchart and / or block diagram.

[0109] Furthermore, the teachings herein can be implemented in the form of a computer program product stored in a storage medium and including a plurality of instructions for causing a computer device to implement the methods described in the embodiments of this disclosure.

[0110] Elements or steps described with respect to one embodiment may be combined with or replaced by elements or steps described with respect to another embodiment. By studying the drawings, disclosure, and appended claims, those skilled in the art can understand and implement variations of the disclosed embodiments in practicing the claimed invention. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite articles "a" or "an" do not exclude multiple. A single processor or other unit may perform the functions of several items recited in the claims. The mere fact that certain measures are recited in dissimilar dependent claims does not indicate that combinations of these measures cannot be advantageously used. Computer programs may be stored or distributed on suitable media, such as optical storage media or solid-state media, supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems. No reference numerals in the claims should be construed as limiting the scope.

Claims

1. A computer-implemented method (100), comprising: Receive (102) data corresponding to an environment-corrected image of an object illuminated by an illumination unit (206), the illumination unit providing time-modulated illumination having a modulation frequency higher than the frame acquisition rate of the imaging device (204) for obtaining an image set, the image set having different spatial intensity modulation modes in each image due to the time-modulated illumination and the imaging device operating in a rolling shutter mode, wherein the environment-corrected image is determined based on the different spatial intensity modulation modes in each image of the obtained image set, and wherein pixel intensity information from each image of the obtained image set is used to construct the environment-corrected image by the following steps: determining the recorded pixel intensity due to ambient light and the recorded pixel intensity due to the time-modulated illumination in each image of the obtained image set from the pixel intensity information of each image of the obtained image set; And based on the recorded pixel intensity caused by the time-modulated illumination in each image of the obtained image set, the environment-corrected image is constructed from the pixel intensity information of each image in the obtained image set; Determine the signal-to-noise ratio (SNR) of at least a portion of the environmental correction image (104); as well as In response to determining that the SNR is lower than the target SNR, an indication of illumination parameters (106) is sent to the illumination unit to increase the amplitude modulation depth of the time-modulated illumination used to illuminate the object when the imaging device acquires a subsequent set of images, the subsequent set of images being used to construct subsequent environmental correction images with an increased SNR for at least a portion of the subsequent environmental correction images.

2. The method of claim 1, wherein the target SNR is based on the skin type of the object.

3. The method of claim 2, wherein the target SNR is associated with a factor A, for which multiple different skin types are associated with obtaining an image having at least the target SNR, wherein the factor A is defined as the ratio of the ambient light level incident on the object to the total amount of light incident on the object, wherein the total amount of light refers to a combination of the ambient light level and the modulation depth of amplitude modulation provided by the illumination unit, and wherein the indication is configured to vary the modulation depth depending on the skin type to obtain the same factor A for each of the different skin types.

4. The method of claim 1, 2 or 3, wherein the indication is configured to control the ambient light level of the object to increase the SNR for at least the portion of the subsequent ambient correction image.

5. The method according to any one of the preceding claims, wherein the indication is configured to cause a reduction in the ambient light level to increase the SNR for at least the portion of the subsequent ambient correction image.

6. The method of claim 5, wherein the indication of the lighting parameters includes control instructions for specifying the ambient light level to be provided by the source of the ambient light.

7. The method of claim 6, further comprising: The instruction is used to control (510) the power supply to the source, thereby providing the specified ambient light level.

8. The method according to any one of the preceding claims, wherein the indication of the lighting parameters includes instructions for indicating that a specified ambient light level is required and / or that a user action is required to provide the specified ambient light level.

9. The method according to any one of the preceding claims, further comprising: When the object is illuminated by the time-modulated illumination, the imaging device (504) acquires the image set of the object; And determine the environment correction image based on the image set of the object.

10. The method according to any one of the preceding claims, further comprising: When the object is illuminated by the time-modulated illumination having an increased amplitude modulation depth, the imaging device (508) acquires the subsequent image set of the object; and determines the subsequent environmental correction image based on the subsequent image set of the object.

11. The method according to any one of the preceding claims, wherein, prior to receiving the data corresponding to the environmental correction image, the initial amplitude modulation depth and / or initial ambient light level of the time-modulated illumination are selected (502) based on the skin type of the object.

12. A tangible machine-readable medium (600) storing instructions (602) that, when executed by at least one processor (604), cause the at least one processor to perform the method according to any one of the preceding claims.

13. An apparatus (700) comprising a processing circuit system (702), said processing circuit system comprising: A receiving module (704) is configured to receive data corresponding to an environment-corrected image of an object illuminated by an illumination unit (206), the illumination unit providing time-modulated illumination having a modulation frequency higher than the frame acquisition rate of the imaging device (204) for obtaining an image set, the image set having different spatial intensity modulation modes in each image due to the time-modulated illumination and the imaging device operating in a rolling shutter mode, wherein the environment-corrected image is determined based on the different spatial intensity modulation modes in each image of the obtained image set, and wherein pixel intensity information from each image of the obtained image set is used to construct the environment-corrected image by the following steps: determining the recorded pixel intensity due to ambient light and the recorded pixel intensity due to the time-modulated illumination in each image of the obtained image set from the pixel intensity information of each image of the obtained image set; And based on the recorded pixel intensity caused by the time-modulated illumination in each image of the obtained image set, the environment-corrected image is constructed from the pixel intensity information of each image in the obtained image set; The determining module (706) is configured to: determine the signal-to-noise ratio (SNR) of at least a portion of the environmental correction image; as well as An indication module (708) is configured to: in response to determining that the SNR is lower than a target SNR, cause an indication of illumination parameters to be sent to the illumination unit to increase the amplitude modulation depth of the time-modulated illumination used to illuminate the object when the imaging device acquires a subsequent set of images, the subsequent set of images being used to construct subsequent environmental correction images with an increased SNR for at least a portion of the subsequent environmental correction images.

14. The device (800) according to claim 13, further comprising a control module (804) for controlling: the lighting unit for providing the time-modulated lighting; and / or a source of ambient light.

15. The device (800) according to claim 13 or 14, further comprising a user interface (806) for providing instructions based on the indication of the lighting parameters to indicate the need for a specified ambient light level and / or the need for user action to provide the specified ambient light level.

Images