Method and corresponding system for determining facial skin color - Patents.com

Abstract

The method for determining the skin colour of a user's area of ​​interest comprises: - importing at least one image (IMk) comprising a representation of the user's area of ​​interest from an image library (LIB) of a user computing device (APP); - processing (TS) the at least one imported image (IMk) using a machine learning model (AI_TS) suitable for providing, for each imported image (IMk), a numerical value (ClrEstm_k) representative of the skin colour of the area of ​​interest present in said at least one imported image (IMk); - performing an evaluation (EVAL) of the skin colour of the area of ​​interest based on the one or more numerical values ​​(ClrEstm_k) of each imported image.

Description

[Technical field]

[0001] Aspects of implementations and embodiments of the present invention relate to methods and corresponding systems for determining facial skin color, particularly in the context of, for example but not limited to, cosmetic product recommendations. [Background technology]

[0002] Purchasing products online, especially via mobile computing devices, is a practical way for consumers to navigate and acquire products. Various techniques exist for providing online shoppers with recommendations of products that may be of interest according to analysis of browsing history, purchase history, comments, product scores, etc.

[0003] However, certain products are difficult to replace face-to-face interactions with online experiences. For example, beauty products such as foundations or other makeup products are difficult to search online and difficult to recommend in an automated manner. This is mainly due to the fact that the consumer characteristics that provide the most plausible basis for recommendations, namely facial skin color in particular, are difficult to estimate in an automated manner. Even if an image or video of a consumer is captured, traditionally, the image or video cannot be used to reliably determine skin color due to technical limitations in image processing due to uncontrolled environments. Inconsistent lighting conditions among various locations result in different color reproductions in the image or video at different locations, and thus determined skin color varies depending on the lighting conditions.

[0004] Patent application 2, filed on July 9, 2020, claiming priority to a provisional patent application 1, filed on July 18, 2019, describes techniques that allow overcoming these technical limitations to accurately estimate skin color in an image, regardless of lighting conditions.

[0005] In particular, the patent application (Patent Document 2) describes a system for generating an automatic estimate of skin color using at least one machine learning model. A user has a mobile computing device equipped with a photo sensor and captures one or more images of their face using the photo sensor. The mobile computing device transmits the one or more images to a device for determining skin color, which determines skin color based on the one or more images using one or more machine learning models.

[0006] For example, a user may capture multiple images by recording video instead of one isolated image, thus modifying lighting conditions by moving the mobile computing device during video capture. When provided with multiple images under various lighting conditions, a machine learning model can be used to generate multiple determinations of skin color, which can then be averaged or otherwise combined to improve the accuracy of the determinations.

[0007] The technique outlined above in the patent application (US Pat. No. 5,399,433) has the drawback that the user must perform the image capture himself, which can sometimes be tedious or even impossible in certain circumstances. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] US Patent Application Publication No. 16 / 516080 [Patent Document 2] PCT / US Patent Application Publication No. 2020 / 041350 Summary of the Invention [Problem to be solved by the invention]

[0009] It would therefore be desirable to limit the effort required for a user to accurately determine their skin color based on images in varying and essentially unknown lighting conditions. [Means for solving the problem]

[0010] In this regard, a method for determining a user's region of interest, e.g. the skin color of the user's face, has been proposed, the method comprising: - importing at least one image from an image library of the user computing device, the image including a representation of the user's region of interest; - processing said at least one imported image with a machine learning model suitable for providing, for each imported image, a numerical value representative of a skin colour of an area of ​​interest present in said at least one imported image; - performing an assessment of skin color of the region of interest based on said one or more numerical values ​​for each imported image. Includes.

[0011] For example, the user computing device may be a smartphone, a tablet computer, a personal computer, a smartwatch, or any other computing device capable of having an image library.

[0012] "Image library" conventionally refers to a directory provided on a user computing device that has access to all or nearly all of the images and photographs stored in the memory of the user computing device or in a remote memory accessible from the user computing device, e.g., a "cloud" type memory.

[0013] The at least one image is preferably a photograph taken using a photo sensor of the user computing device, optionally a photograph taken using another device and imported into the user computing device.

[0014] Surprisingly and unexpectedly, it has been found that machine learning models pre-trained for the techniques outlined above in patent application PCT / US2020 / 041350 achieve better results, i.e., more accurate and truer results, when a user provides multiple images generated from an image library.

[0015] Specifically, the machine learning model has been pre-trained to process images in which lighting conditions are modified by the movement of the mobile computing device during video capture, but has been found to perform even better when processing images in which lighting conditions are more diverse and varied, i.e., images that may be present in the image library of a user computing device.

[0016] Although the present invention is particularly advantageously and beneficially applicable to determining the skin color of the face, the present invention may nevertheless be applied to determining the skin color of another area of ​​interest, e.g. another part of the body, based on said evaluation of the skin color of this area of ​​interest, with particular intent to apply at least one cosmetic product to said area.

[0017] According to one implementation, the at least one image includes metadata that timestamps the creation of each image (i.e., the date and time of capture of the photo), and the assessment of skin color is based on a numerical value of each image weighted according to the current date and the respective timestamp metadata.

[0018] The weighting may correspond to assigning a confidence index to the images according to the date the photo was taken, for example, to give less weight to older photos or more recent photos in assessing skin color.

[0019] According to one implementation, the values ​​of each image are weighted according to the current date and respective timestamp metadata to account for changes in skin tone over the course of the seasons of the year.

[0020] In particular, skin color tone variations, or tanning, may have large amplitudes in certain people and are naturally (i.e., ignoring artificial tanning methods) closely correlated with the season of the year. Thus, aspects of this implementation can provide, for example, a reduced weighting for photos taken in the opposite season to the current date and an increased weighting for photos taken in the same season as the current date.

[0021] Advantageously, weighting taking into account changes in skin colour shade over the course of the seasons of the year is performed such that said assessment of skin colour provides a prediction of what skin colour will be after the current date.

[0022] For example, aspects of this implementation may provide for a reduced weighting of photos taken in seasons prior to the current date season and an increased weighting of photos taken in seasons following the current date season.

[0023] According to one implementation aspect, the at least one image includes metadata that timestamps the creation of each image and / or geolocates the creation of each image (i.e., e.g., GPS coordinates of the location where the photo was captured), and the processing includes pre-processing that includes correcting the color temperature of each image according to the respective metadata.

[0024] In particular, the timestamp metadata and the geolocation metadata allow for inference of lighting conditions with respective color temperatures being compensated for. For example, the timestamp metadata can indicate whether the photo was taken during the day or at night, and the geolocation metadata can further indicate whether the photo was taken in an indoor or outdoor space.

[0025] Advantageously, the correction of the color temperature of each image is further performed according to meteorological archive data corresponding to the respective metadata, making it possible to estimate the color temperature of the lighting conditions of the creation of each image.

[0026] Weather archive data makes it possible to know with certainty for a given date and a given location, in particular the hours of day and night, the duration of sunrise and sunset, and the lighting conditions, which allows for better estimation of lighting conditions and improved color temperature corrections.

[0027] According to one embodiment of the implementation in which the user's region of interest is the user's face, the importing further includes obtaining a reference image representing a reference face of the user, and the processing includes pre-processing including selecting, via facial recognition, the reference face within the at least one image imported from an image library of the user computing device.

[0028] This makes it possible, for example, if an image is imported incorrectly or if it contains multiple faces, to not corrupt the skin color assessment by taking into account different faces of different people during processing.

[0029] For example, obtaining a reference image may include a user's photographic capture of the isolated region of interest, or a user's identification of an image representing the isolated region of interest from among images in the image library.

[0030] According to one aspect of implementation, the machine learning model is a pre-trained convolutional neural network.

[0031] According to one embodiment of implementation, the method further includes recommending at least one cosmetic product based on said assessment of skin color of the region of interest.

[0032] There is also provided a system for determining a skin colour of a region of interest of a user, the system comprising communication means adapted to communicate with a user computing device and configured to import at least one image comprising a representation of the region of interest of the user from an image library of the user computing device, the system comprising processing means adapted to process, for each imported image, the at least one image with a machine learning model suitable to provide a numerical value representative of the skin colour of the region of interest present in the at least one imported image, and further configured to provide an assessment of the skin colour of the region of interest based on the one or more numerical values ​​of each imported image.

[0033] There is also provided a system comprising communication means suitable for communicating with a user computing device and configured to implement the importing step of the method as defined above, and processing means configured to implement the processing and evaluation steps of the method as defined above.

[0034] According to one aspect of the embodiment, the system further includes a user computing device configured to communicate the at least one image of the image library to a communication means.

[0035] According to one aspect of the embodiment, the communication means is suitable for communicating with a user computing device over a telecommunications network such as the Internet.

[0036] There is also provided a computer program comprising instructions which, when the program is executed by a computer, cause the computer to implement a method as defined above.

[0037] There is also provided a computer readable medium containing instructions which, when executed by a computer, cause the computer to implement a method as defined above.

[0038] Other advantages and features of the present invention will become apparent upon consideration of the detailed description of entirely non-limiting embodiments and implementation aspects and the accompanying drawings below. [Brief description of the drawings]

[0039] [Figure 1] 1 illustrates embodiments and implementations of the present invention. [Diagram 2] 1 illustrates embodiments and implementations of the present invention. [Diagram 3] 1 illustrates embodiments and implementations of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0040] FIG. 1 shows an example of a system SYS including a processing means PU configured for processing TS of said at least one image IMi, IMj, IMk using a machine learning model AI_TS and configured to evaluate EVAL the skin colour of a region of interest of a user present in one or more images IMi, IMj, IMk.

[0041] For convenience and as a result of non-limiting arbitrary choice, the case where the set of "said at least one image" consists of multiple images is described. The implementation where the set of "said at least one image" consists of a single image is similar in all respects.

[0042] For the same reasons, we also describe the non-limiting case where the user's region of interest is the user's face.

[0043] The machine learning model is constructed and trained to provide, for each image IMk, a numerical value ClrEstm_k representative of the skin colour of the faces present in each of the images IMk.

[0044] The means for performing the evaluation EVAL is configured to evaluate the facial skin color based on the numerical values ​​ClrEstm_k of each image, for example, the evaluation may include a calculation of combining the values ​​ClrEstm_k, such as a calculation of an average or median, whether weighted or not.

[0045] The communication means COM are suitable for communicating with a user computing device APP and are adapted to receive IN_DAT said images IMi, IMj, IMk from an image library LIB of the user computing device APP.

[0046] In particular, the user computing device APP may belong to the system SYS in the sense that it may be specifically configured to cooperate with the processing means PU, for example via software, via the execution of an application or website, with the aim of transmitting images IMi, IMj, IMk during import IN_DAT of images by the processing means PU.

[0047] The user computing device APP and the communication means COM can communicate using any suitable communication technology, such as wireless communication technologies, e.g. Wi-Fi, Wi-MAX, Bluetooth, 2G, 3G, 4G, 5G and LTE, or wired communication technologies, e.g. Ethernet, FireWire and USB. In particular, the user computing device and the device for determining skin color can communicate at least partially via the Internet.

[0048] The user computing device APP may be, for example, a smartphone, a tablet computer, a personal computer, a smartwatch or any other computing device capable of having an image library LIB.

[0049] The image library LIB is conventionally a directory provided at the interface of the user computing device APP, which has access to all or almost all of the images and photos IM1 to IM4, IMk stored in the internal non-volatile memory INT_NVM of the user computing device APP. The images and photos IM1 to IM4, IMk of the library LIB may also or instead be stored in a remote memory (for example, in the "cloud", using the familiar expression) of a server CLD accessible from the user computing device APP.

[0050] The at least one image IM1 to IM4, IMk is preferably a facial photograph taken using a photo sensor CAM of the user computing device APP or a facial photograph taken using another device and imported into the image library LIB of the user computing device APP.

[0051] The images IM1-IM4, IMk in the library of images IMk have a digital image data format, such as JPEG format (JPEG stands for Joint Photographic Expert Group), PNG format (PNG stands for Portable Network Graphics), GIFF format (GIFF stands for Graphics Interchange Format), TIFF format (TIFF stands for Tagged Image File Format) or any other image format.

[0052] Advantageously, each image IM1 to IM4, IMk of the library of images IMk further comprises metadata MTD1 to MTD4, MTDk providing various information relating to the image.

[0053] In particular, at least certain images of the images IM1 to IM4, IMk of the library LIB comprise timestamp metadata providing information regarding the date and time of creation of the respective image and / or geolocation metadata providing information regarding the place of creation of the respective image.

[0054] As will be seen with reference to FIG. 2 below, the timestamp metadata and / or geolocation metadata may enable refinement and improvement of the processing by the machine learning model AI_TS and the skin color assessment EVAL.

[0055] The skin color thus obtained by the machine learning model AI_TS is true to life and can later be used to recommend one or more cosmetic products that closely correlate with the skin color of each user, in particular foundations, powders and variants of foundations and powders, as well as cosmetic products that will complement or be suitable for the user's skin color.

[0056] Now, reference is made to FIG.

[0057] FIG. 2 illustrates an example of a method for determining facial skin color, implemented in particular by the processing means PU of the system described with reference to FIG.

[0058] The method comprises importing IN_DAT at least one image IMk from an image library LIB of a user computing device in a manner as described with reference to FIG.

[0059] The processing TS of the at least one image IMk implemented in this example includes a pre-processing P_TS and a processing using a machine learning model AI_TS configured to provide a numerical value ClrEstm_k representative of the skin color of the face present in the at least one image IMk.

[0060] The pre-processing P_TS comprises converting the image IMk into a format suitable for the machine learning model AI_TS. In particular, the pre-processing P_TS is suitable for isolating, via a conventional face detection mechanism FceDet known per se, one or more parts of the image containing a face, and for centering and scaling Cntr+Scl the parts of the image containing the face in order to provide normalized data to the machine learning model.

[0061] In essence, centering Cntr may involve cropping the image to preserve only the portion containing the face, and scaling Scl involves enlarging or shrinking the cropped image, and undersampling or oversampling the pixels of the cropped image to provide a cropped image having a set size and a set resolution.

[0062] Advantageously, the pre-processing further comprises selecting cropped images containing the same face, via a conventional face recognition mechanism FceReco, known per se.

[0063] The face recognition FceReco identifies and detects the face of interest, i.e. the reference face provided in the reference image IMref. Advantageously, the reference face is isolated in the reference image IMref, i.e. the reference face is the only face present in the reference image IMref.

[0064] The reference image IMref may be, for example, imported from an image library LIB of the user computing device and identified by the user as the reference image IMref. Alternatively, the reference image IMref may be a self-taken face photograph (conventional "selfie") PhtSlf that the user may take with a photo sensor (CAM) of the user computing device, advantageously to include the user's face in isolation as the reference face.

[0065] As described above with reference to figure 1, the images IMk of the image library advantageously comprise metadata MTDk. In this case, the pre-processing P_TS can advantageously comprise a correction TempCorr of the colour temperature of the imported images, the correction being established according to the respective metadata MTDk.

[0066] Color temperature is a quantity well known to those skilled in the art, usually measured in units of Kelvin, that characterizes a light source by comparison with the theoretical principles of thermal radiation of a black body.

[0067] Advantageously, the correction of the color temperature of each image is further performed according to external data EXT_MTD, such as meteorological archive data and geographical data, making it possible to estimate the color temperature of the lighting conditions of the creation of each image, corresponding to the respective metadata MTDk.

[0068] In particular, the timestamp metadata MTDk ts and the geolocation metadata MTDk geoloc allow for inferring the lighting conditions with the respective color temperatures to be compensated for. For example, the timestamp metadata ts can indicate whether the photo was taken during the day or at night, and the geolocation metadata geoloc can further indicate whether the photo was taken in an indoor or outdoor space.

[0069] Weather archive data makes it possible to know with certainty for a given date and a given location, in particular the hours of day and night, the duration of sunrise and sunset, and the lighting conditions, which allows for better estimation of lighting conditions and improved color temperature corrections.

[0070] The correction of the color temperature TempCorr by the preprocessing means P_TS can for example use a list of conditions that may be satisfied by the metadata MTDk and, optionally, by the external data EXT_MTD indexed by said metadata MTDk.

[0071] A look-up table can optionally select a particular color temperature correction depending on which conditions are met.

[0072] For example, among the simple possible conditions, one can mention "dawn", "daytime", "nighttime", "evening", "indoors", "outdoors", "clear", "cloudy", "rain", "snow", etc. Other and more complex conditions can also be imagined and envisaged in this regard.

[0073] For example, if the metadata MTDk reflects the conditions "night" and "indoors", then color temperature corrections may be performed to correct for "indoor lighting" or "incandescent" type lighting. If the metadata MTDk reflects the conditions "evening", "outdoors" and "daylight", then color temperature corrections may be performed to correct for sunset type lighting.

[0074] The color temperature correction TempCorr mechanism is equivalent to the debayering or demosaicing technique, which is conventionally used to individually rebalance the RGB channels of a RAW image captured by a photographic sensor before the image is stored.

[0075] Therefore, the mechanism of color temperature correction TempCorr advantageously uses a debayering or demosaicing matrix in which points are individually adjusted in the imported image IMk according to conditions evaluated via the metadata MTDk. These adjustments are then integrated into the still image file and verified on the premise that the color of one element in the image is always known (for example the white of the eyes or, to a lesser extent, the white of the teeth).

[0076] The image IMk thus centered Cntr on the face FceDet, scaled Scl, optionally selected FceReco, and optionally corrected TempCorr is provided to the machine learning model AI_TS.

[0077] The machine learning model AI_TS calculates a numerical value ClrEstm_k that represents the skin color of the face present in each image IMk.

[0078] For example, the machine learning model AI_TS can be implemented via a pre-trained convolutional neural network, for example in a manner as described in patent application PCT / US2020 / 041350 or as outlined below with reference to FIG. 3. The machine learning model AI_TS can also be a forward propagation neural network or a recurrent neural network. Any suitable training technique can be used, in particular gradient descent techniques such as stochastic gradient descent, batch gradient descent and mini-batch gradient descent.

[0079] Furthermore, the machine learning model AI_TS may be capable of selecting suitable images ImSel by detecting abnormal image capture conditions in the images, particularly unsuitable image capture angles, etc.

[0080] This selection of suitable images can be done in two stages. First, it allows the machine learning model to explicitly test that conditions are met (e.g., the face is looking at the photo sensor, there is only one face in the image, the face is actually in the image). After this first filtering stage, a second selection is made using the quality score wght1, which is learned, for example, in a weakly supervised way during the training of the model AI_TS. The quality score wght1 is purely statistical and specifically optimized to improve the accuracy of the model AI_TS during training.

[0081] In this way, the machine learning model AI_TS can assign a first weight wght1 to each result, making it possible to discard particularly anomalous results.

[0082] Once the machine learning model AI_TS provides a value ClrEstm_k for each image IMk, an evaluation EVAL of the facial skin color is made based on said value ClrEstm_k and the respective first weights wght1.

[0083] For example, the evaluation EVAL may include a calculation that combines the values ​​ClrEstm_k, such as calculating a weighted average or median value with a first weight wght1.

[0084] Furthermore, the assessment can be advantageously weighted by a second weight wght2, which is obtained based on the metadata MTDk of the respective image, and in particular the timestamp metadata ts, to take into account changes in skin colour shade over the course of the seasons of the year, i.e. skin tanning.

[0085] In particular, it may be desirable to select photographs taken during the same season as the current date, i.e., the date for which facial skin color is to be determined using the disclosed techniques.

[0086] For example, the weight of images whose dates are in the opposite season to the current date can be reduced using a lower second weight wght2, while the weight of images whose dates are in the same season as the current date can be increased using a higher second weight wght2.

[0087] Advantageously, a second weight wght2 may be determined such that said evaluation EVAL of skin colour provides a prediction of what the skin colour will be after the current date.

[0088] For example, this can be achieved by decreasing the weighting of images whose dates are in a season prior to the season of the current date, and increasing the weighting of images whose dates are in a season following the season of the current date.

[0089] Measuring skin color as a function of season using a spectrophotometer (or "spectrophotometer") reveals that there is a change in the average skin tone in summer compared to winter. These measurements make it possible to establish the macroscopic variation in the population and are integrated into the model of evaluation EVAL with the intention of determining said second weight wght2.

[0090] FIG. 3 illustrates a non-limiting example of an implementation of a training TRN of a machine learning model AI_TS intended to provide a numerical value ClrEstm_k representative of the skin color of a face present in at least one image IMk, as described above with reference to FIGS. 1 and 2.

[0091] The training can be implemented by a processing means PU which comprises an additional training means TRN, which is arranged in particular to control means for receiving data IN_DAT, pre-processing means P_TS, a machine learning model AI_TS and an evaluation means EVAL.

[0092] A set of training images IN_DAT associated with factual skin color information GRND_TRTH, known as “ground truth”, is collected to implement training TRN of the machine learning model AI_TS.

[0093] The ground truth for skin color, GRND_TRTH, is used as empirical evidence or information for labeling images provided by training subjects, i.e., volunteers called panelists.

[0094] For example, the ground truth GRND_TRTH may be collected by a user using industry standard techniques for determining skin color, such as comparison to a color chart or evaluation with a spectrophotometer dedicated to measuring skin color.

[0095] In comparison to a colour chart, a portion of the image may contain a representation of a known reference colour chart, and colour correction of the image to return the representation of the reference colour chart to its original colour may enable a ground truth GRND_TRTH for the skin colour present in the image after said colour correction to be determined.

[0096] For example, the color correction providing the ground truth GRND_TRTH can be implemented in the preprocessing P_TS as commanded by the training means TRN.

[0097] The spectrophotometer may be used at least once on the panelist's skin, and the color measured by the spectrophotometer is used directly as the ground truth GRND_TRTH, possibly with some color code conversion.

[0098] The set of training images IN_DAT is provided by the panelists via a user computing device of the same type as device APP described above with reference to FIG.

[0099] For example, the training images IN_DAT are imported via the use of a photosensitive sensor CAM of the panelist's user computing device to capture one or more training images including the panelist's face. The training images can be drawn from, for example, individual photos PhtSlf taken in selfie mode or at least one video recording Vid360 taken in selfie mode, where the lighting conditions can be modified by moving the viewpoint, for example by rotating the device around the panelist's face.

[0100] A number of training images can then be generated by extracting individual images from the video Vid360.

[0101] Capturing video from which multiple training images can be derived can be advantageous, at least because it greatly increases the efficiency of generating large amounts of training data under a variety of lighting conditions.

[0102] If necessary, those skilled in the art can refer to the description of training the machine learning model described in patent application PCT / US2020 / 041350.

[0103] It should be noted that during the implementation of the training process, the pre-processing P_TS only includes face detection FceDet and centering and scaling Cntr+SCl as described above with reference to Fig. 2. In particular, it can be assumed that the panelists provide images of only their own faces, in which case face recognition FceReco is not required. Furthermore, since the training images are derived from photographs or videos, the metadata does not provide a specific context that would allow parameterizing the correction of the color temperature TempCorr.

[0104] Additionally or alternatively, optionally, the training images IN_DAT may be drawn from an image library (IMk∈LIB) of the panelist's user computing device, which may then be adapted to perform the face recognition FceReco step and the color temperature correction TempCorr step as described with reference to FIG.

[0105] Therefore, it has been found, surprisingly, that training using only photographs and video recordings as described above, or indeed as described in patent application PCT / US2020 / 041350, provides very satisfactory results in the implementation described with reference to FIG. 2, i.e. inference of the machine learning model AI_TS using images generated from the image library LIB.

[0106] In the training implementation, the machine learning model AI_TS performs a parameterizable calculation on the training images IN_DAT to provide a numerical value ClrEstm representing the skin color of the faces present in the training images.

[0107] The assessment of skin colour EVAL can be performed as described with reference to FIG. 2 via a statistical operation AVRG which combines the various values ​​ClrEstm.

[0108] In order to reparameterize the calculations performed by the machine learning model AI_TS, the skin color thus evaluated EVAL or the actual numerical value ClrEstm is then directly compared with the ground truth GRND_TRTH to come as close as possible to the ground truth.

[0109] In other words, the ground truth for skin color GRND_TRTH is used as labeling data to display the results of the desired processing of the training image IN_DAT by the machine learning model AI_TS.

[0110] It should be noted that the machine learning model AI_TS is advantageously trained using data of multiple panelists. In this respect, the training images IN_DAT and the ground truth GRND_TRTH associated with each panelist can be stored in the memory of the processing means PU and the training means TRN. The calculations of the machine learning model AI_TS are parameterized for all data of all panelists so as to be as universal as possible.

[0111] In summary, to determine the facial skin color, the processing means PU uses the machine learning model AI_TS as described above to determine the facial skin color present in at least one image imported from the image library of the user computing device APP, which can then be used to recommend one or more cosmetic products that compensate for the thus determined skin color or that are suitable for the skin color. [Explanation of symbols]

[0112] AI_TS Machine Learning Model APP Device CAM Photo Sensor CLD Server COM communication means EVAL Evaluation method EXT_MTD External Data GRND_TRTH Ground Truth IMref Reference Image ImSel compatible images IN_DAT Import INT_NVM Internal non-volatile memory LIB Library MTD1~MTD4, MTDk metadata P_TS Pre-processing means PhtSlf Photo PU Processing Means RES Telecommunications Network Scl Scaling SYS system TRN training tools TS Processing Vid360 Video wght1 quality score

Claims

1. 1. A method for recommending cosmetics according to a skin color of a user's area of ​​interest, comprising: - importing at least one image (IMk) comprising a representation of said region of interest of said user from an image library (LIB) of a user computing device (APP); - processing (TS) of at least one imported image (IMk) using a machine learning model (AI_TS) suitable for providing, for each imported image (IMk), a numerical value (ClrEstm_k) representative of the skin colour of the region of interest present in said at least one imported image (IMk); making an assessment (EVAL) of the skin colour of the region of interest based on the one or more values ​​(ClrEstm_k) of each imported image; Including, said images (IMk) contain metadata (MTDk,ts) that timestamp the creation of said respective image; - said assessment of skin colour (EVAL) is based on said numerical values ​​(ClrEstm_k) of each image weighted (wght2) according to the current date and the respective timestamp metadata (ts); A method according to claim 1, wherein the recommending at least one cosmetic product is based on the evaluation (EVAL) of the skin color of the region of interest.

2. The method of claim 1 , wherein the region of interest of the user is the user's face.

3. 2. The method of claim 1, wherein the numerical values ​​(ClrEstm_k) for each image are weighted (wght2) according to the current date and respective timestamp metadata (ts) to account for changes in the skin color shade over the course of the seasons of the year.

4. 4. The method of claim 3, wherein the weighting (wght2) taking into account the changes in shade of the skin color over the course of the seasons of the year is performed such that the evaluation (EVAL) of the skin color provides a prediction of what the skin color will be after the current date.

5. said at least one image (IMk) comprises metadata (MTDk,ts) time-stamping the creation of said respective image and / or metadata (MTDk,geoLoc) geolocating the creation of said respective image; Method according to claim 1 or 4, wherein said processing (TS) comprises a pre-processing (P_TS) comprising a correction (TempCorr) of the color temperature of each image (IMk) depending on said respective metadata (MTDk).

6. 6. The method according to claim 5, wherein the correction (TempCorr) of the color temperature of each image is further performed according to meteorological archive data (ArchMeteo) corresponding to said respective metadata (MTDk), making it possible to estimate the color temperature of the lighting conditions of the creation of said respective image.

7. said importing further comprises obtaining a reference image (IMref) representative of a reference face of said user; The method of claim 1 or 6 in combination with claim 2, wherein the processing includes a pre-processing (P_TS) that includes selecting the reference face in the at least one image (IMk) imported from the image library of the user computing device via face recognition (FceReco).

8. 8. The method of claim 7, wherein obtaining the reference image (IMref) comprises a photo capture (PhtSlf) of the user of the isolated area of ​​interest, or identification by the user of an image representative of the isolated area of ​​interest among the images in the image library (IMk).

9. The method according to any one of claims 1 to 8, wherein the machine learning model (AI_TS) is a pre-trained (TRN) convolutional neural network.

10. 1. A system for determining skin color of a region of interest of a user, comprising: communication means (COM) suitable for communicating with a user computing device (APP) and adapted to import, from an image library (LIB) of said user computing device (APP), at least one image (IMk) comprising a representation of said area of ​​interest of said user; processing means (PU) configured to perform a processing (TS) of at least one imported image (IMk) using a machine learning model (AI_TS) suitable for providing, for each imported image (IMk), a numerical value (ClrEstm_k) representative of the skin colour of the region of interest present in said at least one imported image (IMk); Including, said processing means (PU) are further adapted to perform an assessment (EVAL) of said skin colour of said region of interest based on said one or more numerical values ​​(ClrEstm_k) of each imported image; said processing means (PU) are further adapted to perform an assessment (EVAL) of said skin colour of said region of interest based on said numerical values ​​(ClrEstm_k) of an imported image; said communication means (COM) being further adapted to import said plurality of images (IMk) including metadata (MTDk, ts) time-stamping the creation of each said image; said processing means (PU) is further adapted to perform said assessment (EVAL) of said skin colour based on said numerical values ​​(ClrEstm_k) of each image weighted (wght2) according to a current date and a respective timestamp metadata (ts), The system, wherein said processing means (PU) is configured to recommend at least one cosmetic product based on said evaluation (EVAL) of said skin colour of said region of interest.

11. The system of claim 10, wherein the communication means (COM) is further configured to implement the importing step of the method of any one of claims 2 to 9, and the processing means (PU) is further configured to implement the processing (TS) and evaluation (EVAL) steps of the method of any one of claims 2 to 9.

12. The system according to claim 10 or 11, further comprising said user computing device (APP) adapted to communicate said at least one image (IMk) of said image library (LIB) with said communication means (COM).

13. System according to any one of claims 10 to 12, wherein said communication means (COM) are suitable for communicating with said user computing device (APP) via a telecommunications network (RES).

14. A computer program comprising instructions which, when said program is executed by a computer, cause said computer to implement a method according to any one of claims 1 to 9.

15. A computer readable medium comprising instructions which, when executed by a computer, cause the computer to implement the method of any one of claims 1 to 9.

Images