Method for predicting the current or future physical characteristics of a part of a human body and associated devices

Abstract

The present invention relates to a method for predicting the current or future physical characteristics of a part of a human body by a device, the part comprising skin or hair. The device is able to acquire images of said parts and apply a digital model on these to predict a state of the physical characteristics of the parts of a human body, the method comprising the following phases: - a first use of the device, - an analysis comprising the steps of: - collecting the data originating from the use, - determining whether the collected data meet a triggering criterion to obtain a determination result, - triggering an update phase according to the determination result, - updating the digital model, - second use of the device.

Description

[0001] Method for predicting the current or future physical characteristics of a part of a human body and associated devices

[0002] The present invention relates to a method for predicting the current or future physical characteristics of a part or portion of a human body. The present invention also relates to associated computer program product and readable information medium.

[0003] In the cosmetics industry, it is useful to be able to offer to the clients products that are suited to their needs quickly and accurately.

[0004] To address this need, it is known to use apparatuses intended to examine portions of the body of a user such as the hair or the skin, to guide the user towards a set of products suited to the needs of these portions.

[0005] For this purpose, the apparatus implements an algorithm obtained by artificial intelligence. More specifically, the algorithm uses a set of data collected on the user and inputs them into a digital model to determine a characteristic of the portion, for example his / her tone. On the basis of this determined characteristic, it is then possible to suggest specific products to the user.

[0006] However, in use, it turns out that these algorithms are not as accurate as expected, leading to suggestions that are not suited to the user.

[0007] Hence, there is a need for a method for predicting the current or future physical characteristics of a portion of a human body which allows obtaining a better prediction.

[0008] T o this end, an object of the present description is a method for predicting the current or future physical characteristics of a portion of a human body by a device, the portion comprising skin or hairs and the device being able to acquire images of the portion and to apply a digital model on at least one portion of the acquired images to obtain a predicted value of the current or future physical characteristics of a portion of a human body, the digital model being obtained by machine learning, the method comprising the following phases:

[0009] - a first phase of using the device comprising the steps of:

[0010] - acquiring images of the skin or the hair of a user,

[0011] - applying the digital model on at least one portion of the acquired images to obtain a predicted value of the current or future physical characteristics of a portion of a human body,

[0012] - an analysis phase comprising the steps of:

[0013] - collecting the data of the first phase,

[0014] - determining whether the collected data meet or not at least one triggering criterion, to obtain a determination result, - triggering an update phase according to the determination result,

[0015] - a phase of updating the digital model, the update phase comprising the steps of:

[0016] - training the digital model at least on the collected data to obtain a new digital model, and

[0017] - a second phase of using the device comprising the steps of:

[0018] - acquiring images of the skin or the hair of the user,

[0019] - applying the new digital model on at least one portion of the acquired images to obtain a value of the current or future physical characteristics of a portion of a human body.

[0020] Thus, the method allows obtaining predicted physical characteristics with an accuracy increasing at each re-training.

[0021] According to other advantageous aspects of the invention, the method comprises one or more of the following features considered separately or according to any feasible combination:

[0022] - the method comprises a step of determining the distance between the collected data with used data, the method further comprises a step of correcting the collected data when the distance is greater than or equal to a threshold value, to obtain corrected data, the training step being implemented on the corrected data. By correcting the data before re-training, this allows obtaining data that allow for a better convergence towards a more accurate model. a triggering criterion is a time interval or a number of collected data.

[0023] - the method includes a step of collecting the satisfaction of the user, a triggering criterion being the client satisfaction percentage. a triggering criterion is a comparison criterion between the collected data and expected data.

[0024] - the expected data are the values of the actual current or future physical characteristics, the comparison criterion being a distance criterion between the collected data and the expected data.

[0025] - the expected data are the training data, the comparison criterion being a distance criterion between the collected data and the expected data.

[0026] Each of these triggering criteria allows carrying out a re-training suited to the specific case in order to ensure that the re-training takes place when it is really useful.

[0027] An object of the present description is a computer program product including a readable information medium, on which a computer program is recorded comprising program instructions, the computer program could be loaded on a data processing unit and adapted to cause the implementation of at least one step of a method as described before when the computer program is implemented on the data processing unit. An object of the present description is a readable information medium including program instructions forming a computer program, the computer program could be loaded on a data processing unit and adapted to cause the implementation of at least one step of a method as described when the computer program is implemented on the data processing unit.

[0028] The invention will appear more clearly upon reading the following description, given only as a non-limiting example and made with reference to the drawings wherein:

[0029] Figure 1 is a schematic view of a set including a device for predicting the current or future physical characteristics of a portion of a human body, an individual using the device and a server,

[0030] Figure 2 is a detailed view of the prediction device of Figure 1 , and

[0031] Figure 3 is a flowchart of the steps of a method for predicting the current or future physical characteristics of a portion of a human body by the device of Figure 2.

[0032] A prediction device 10 interacting with its environment, namely an individual 12 and a server 14 is shown in Figure 1.

[0033] In general, like the present case, the individual 12 is the user of the prediction device 10.

[0034] The prediction device 10 is able to predict current or future physical characteristics of a portion of the body of the individual 12 by implementation of a prediction method that will be described later on.

[0035] The characteristics are characteristics that are useful to determine the products suited to the individual 12, whether these are skin care products, makeup products or hair care products.

[0036] The characteristics are characteristics relating to the portion of the body of the individual 12 or characteristics relating to the expected state of the portion of the body of the individual 12 after application of a cosmetic product

[0037] Specific examples of characteristics include the presence of wrinkles or not, the pigmentation of the portion, the hydration of the portion, the wrinkle score representative of the degree of seriousness of the wrinkles, the color of the skin, the visible pores, the diameter of the hairs, the percentage of white hairs or the hair color.

[0038] For example, the prediction device 10 is a smartphone, a tablet or a computer.

[0039] Referring to Figure 2, the prediction device 10 comprises an acquisition module 16, a processing module 18 and an output module 20.

[0040] The acquisition module 16 is able to acquire images of the individual 12.

[0041] More specifically, the acquisition module 16 acquires images of a portion of the individual 12. According to the described example, the acquisition module 16 is a camera.

[0042] In addition, the acquisition module 16 allows acquiring an image in the form of a file enabling processing of the acquired images.

[0043] The processing module 18 is able to process the images to predict the physical characteristics.

[0044] The operations applied on the images by the processing module 18 consist of digital operations which will be described later on.

[0045] The processing module 18 includes several units, namely a collection unit 22, an application unit 24, a triggering unit 26 and a correction unit 28.

[0046] These units implement the corresponding steps of the prediction method.

[0047] The output module 20 is able to inform the user of the results of the processing module 18.

[0048] For example, the output module 20 is a screen on which the predicted characteristic is displayed.

[0049] The prediction device 10 is in communication with the server 14.

[0050] In particular, these communications allow carrying out updates of the prediction device 10.

[0051] The operation of the prediction device 10 is now described with reference to Figure 3 which illustrates an example of implementation of a prediction method.

[0052] The prediction method includes a first use phase P1, an analysis phase P2, an update phase P3 and a second use phase P4.

[0053] The first use phase P1 includes an acquisition step 100 and an application step 110.

[0054] During the acquisition step 100, the acquisition module 16 acquires images of the portion of the individual 12.

[0055] During the application step 110, the processing module 18, and more specifically the application unit 24, applies a digital model on at least one portion of the acquired images.

[0056] The digital model takes as input at least one portion of the acquired images, preferably all of the acquired images, and gives as output a prediction of the current or future physical characteristics of a portion of the human body of the individual 12.

[0057] For example, the used digital model is a neural network.

[0058] In general, the neural network includes an ordered sequence of neuron layers each of which takes its inputs from the outputs of the previous layer.

[0059] More specifically, each layer comprises neurons taking their inputs from the outputs of the neurons of the preceding layer, or from the input variables for the first layer.

[0060] Alternatively, more complex neural network structures may be considered with a layer which can be connected to a more distant layer than the immediately previous layer. An operation, i.e. a type of processing, to be performed by said neuron in the corresponding processing layer is also associated with each neuron.

[0061] Each layer is connected to the other layers by a plurality of synapses. A synaptic weight is associated with each synapse, and each synapse forms a link between two neurons. It is often a real number, which takes both positive and negative values. In some cases, the synaptic weight is a complex number.

[0062] Each neuron is capable of performing a weighted sum of the value(s) received from the neurons of the preceding layer, each value then being multiplied by the respective synaptic weight of each synapse, or connection, between said neuron and the neurons of the preceding layer, then applying an activation function, typically a non-linear function, to said weighted sum, and outputting from said neuron, in particular to the neurons of the following layer which are connected thereto, the value resulting from the application of the activation function. The activation function allows introducing a non-linearity into the processing performed by each neuron. The sigmoid function, the hyperbolic tangent function, the Heaviside function are examples of activation functions.

[0063] As on optional supplement, each neuron is also capable of also applying a multiplying factor, also so-called bias, to the output of the activation function, and the value output from said neuron is then the product of the bias value and the value obtained from the activation function.

[0064] Such a neural network is trained on a database including images and the prediction of each physical characteristic to be predicted.

[0065] As the case might be, the database is more or less large, and could in particular reach a few thousands images to one billion images.

[0066] Upon completion of the first phase P1 , the processing module 18 has predictions for a set of images specific to the individual 12.

[0067] The second phase P2 includes a collection step 200, a determination step 210 and a triggering step 220.

[0068] During the collection step 200, the collection unit 22 collects the data of the first phase P1.

[0069] The data of the first phase P1 include both the images acquired by the acquisition module 16 and the values predicted for the physical characteristics.

[0070] Thus, the collection unit 22 collects all these data.

[0071] For this purpose, according to one embodiment, the collection unit 22 communicates with the processing module 18.

[0072] During the determination step 210, the triggering unit 26 determines whether or not the collected data meet at least one triggering criterion. As it will appear in the following description, the triggering criterion allows determining whether a re-training of the digital model will be performed or not.

[0073] Many examples could be considered for the triggering criterion / criteria. Some particularly advantageous examples are detailed hereinafter, bearing in mind that these examples could be combined together.

[0074] According to one example, the triggering criterion is a time interval.

[0075] One could decide that once three months have elapsed, a re-training of the digital model will be performed.

[0076] Another example of a triggering criterion is a number of collected data.

[0077] For illustration, one could select a number of threshold data equal to 100,000 collected images.

[0078] Client satisfaction may also be a triggering criterion.

[0079] For this purpose, the opinion of the users is collected after use of the prediction device 10, for example, in the form of a survey or a rating.

[0080] The results thus collected are then sent to the server 14 and an overall rating will allow determining whether the users are more or less satisfied with the predictions provided by the prediction device 10 and, consequently, re-training or not the digital model.

[0081] The user may also be requested when comparing the collected data and the expected data, which could be another triggering criterion. It could be suggested to the user to notice whether the collected data correspond to his / her expectations through a rating. The collected data could vary according to the environment and therefore generate errors like a photograph captured in an excessively bright location such as a cosmetics shop.

[0082] The user could then confirm, or not, that the brightness is suitable and an average rating that is too low for all users will then imply a re-training of the digital model.

[0083] In another embodiment, the triggering criterion is a criterion relating to the comparison of the collected data with the training data of the digital model.

[0084] Typically, the distance between the collected data and the training data of the digital model will be compared with a threshold.

[0085] The error probability or the accuracy are examples of such distances.

[0086] In the presence of an excessively great distance, a re-training will be performed.

[0087] During the triggering step 220, the triggering unit 26 triggers the update phase according to the result of the determining step 210 assuming herein that the triggering criterion is met.

[0088] In the event that the triggering criterion is not met, the collection unit 22 continues collecting data and implements the determination step again. Thus, the processing module 16 implements the collection 200 and determination 210 steps until the triggering criterion is met.

[0089] The phase of updating the digital model P3 aims to update the digital model to obtain a more effective digital model.

[0090] According to the described example, the update phase P3 includes a determination step 300, a correction step 310 and a training step 320.

[0091] During the first determination step 300, the correction unit 28 determines the distance between the collected data and the training data of the digital model.

[0092] For example, the distance is the color, the brightness, the resolution or the satisfaction of the client.

[0093] This distance may be determined digitally via computation algorithms or by proposing a rating system to the user before the acquired image is used by the digital model.

[0094] When the distance is greater than a predetermined threshold, the correction step 310 is implemented.

[0095] During this correction step 310, the correction unit 28 corrects the collected data so that the obtained data feature a distance to the training data below the predetermined threshold.

[0096] For example, it could be selected that each collected image should have the same average luminosity over the pixels, the brightness being changed when the image does not meet this criterion.

[0097] Thus, a set of corrected data is obtained.

[0098] During the training step 320, the server 14 proceeds with a re-training of the digital model at least on the corrected data to obtain a new digital model.

[0099] This allows refining the predictions of the prediction device 10, the new model being, by construction, more suited to the corrected data.

[0100] In some embodiments, re-training of the digital model is performed on all data.

[0101] The second use phase P4 includes an acquisition step 400 and an application step 410.

[0102] The acquisition 400 and application 410 steps are similar to the previously-described acquisition 100 and application 110 steps except that the used digital model is different, the digital model being the digital model obtained upon completion of the implementation of the update phase P3.

[0103] The predictions of the prediction device 10 are then more accurate than the predictions obtained during the first use phase P1.

[0104] This increase in accuracy is obtained in a simple way by data collection and retraining of the used digital model. In particular, this allows adapting the digital model to the new conditions of use of the digital model with a good robustness.

[0105] As a specific example, a digital model developed for a specific population is not, a priori, suited for another population. With the present method, the digital model will progressively provide predictions suited to the other population. In that sense, the digital model becomes a local model.

[0106] Improvements in hardware or in conditions of use (insolation) could also justify such a re-training.

[0107] The method also allows solving the problem of non-availability of training data by progressively enlarging the training base or, at least, by making it increasingly relevant with regards to the desired prediction.

[0108] Other variants of the described method may be considered.

[0109] For example, no correction step is implemented, the re-training data consisting directly of the collected data. In such a case, the processing module 18 does not include any correction unit 28.

[0110] According to the described example, the re-training is performed remotely on the server 14 but it could also be considered to carry out the re-training locally. This could enable a customization of the digital model.

[0111] Moreover, other hardware implementations may be considered.

[0112] In the example of Figure 2, each of the collection unit 22, the application unit 24 and the triggering unit 26 as well as the correction unit 28, as an optional supplement, is made in the form of a software, or a software brick, executable by the processor. The memory of the electronic device for predicting the current or future physical characteristics of a portion of a human body is then able to store an image collection software, a digital model application software and a software for triggering re-training of a digital model, as well as a software for correcting the collected data as an optional supplement. The processor is then able to execute each of the software among the image collection software, the digital model application software and the software for triggering and re-training a digital model, as well as a collected data correction software, as an optional supplement.

[0113] In a variant that is not shown, each of the collection unit 22, the application unit 24 and the triggering unit 26 as well as the correction unit 28, as an optional supplement, is made in the form of a programmable logic component, such as an FPGA (standing for Field Programmable Gate Array in English), or an integrated circuit, such as an ASIC (standing or Application Specific Integrated Circuit in English).

[0114] When the electronic device for predicting the current or future physical characteristics of a portion of a human body is made in the form of one or several software, i.e. in the form of a computer program, also so-called computer program product, it could also be recorded on a computer-readable medium, not shown. For example, the computer-readable medium is a medium able to record electronic instructions and to be coupled to a bus of a computer system. For example, the readable medium is an optical disk, a magneto-optical disk, a ROM memory, a RAM memory, any type of non-volatile memory (for example FLASH or NVRAM) or a magnetic card. A computer program comprising software instructions is then recorded on the readable medium.

Claims

CLAIMS1. A method for predicting the current or future physical characteristics of a part of a human body by a device, the part comprising skin or hairs, the device being able to acquire images of the part, the device being able to apply a digital model on at least one part of the acquired images to obtain a predicted value of the current or future physical characteristics of a part of a human body, the digital model being obtained by machine learning, the method comprising the following phases:- a first phase of using the device comprising the steps of:- acquiring images of the skin or the hair of a user,- applying the digital model on at least one part of the acquired images to obtain a predicted value of the current or future physical characteristics of a part of a human body,- an analysis phase comprising the steps of:- collecting the data of the first phase,- determining whether the collected data meet or not at least one triggering criterion, to obtain a determination result,- triggering an update phase according to the determination result,- a phase of updating the digital model, the update phase comprising the steps of:- training the digital model at least on the collected data to obtain a new digital model, and- a second phase of using the device comprising the steps of:- acquiring images of the skin or the hair of the user,- applying the new digital model on at least one part of the acquired images to obtain a value of the current or future physical characteristics of a part of a human body.

2. The method according to claim 1 , wherein the method further includes:- a step of determining the distance between the collected data with used data,- a step of correcting the collected data when the distance is greater than or equal to a threshold value, to obtain corrected data, the training step being implemented on the corrected data.

3. The method according to claim 1 or 2, wherein a triggering criterion is a time interval or a number of collected data.

4. The method according to any one of claims 1 to 3, wherein the method includes a step of collecting the satisfaction of the user, a triggering criterion being the client satisfaction percentage.

5. The method according to any one of claims 1 to 4, wherein a triggering criterion is a comparison criterion between the collected data and expected data.

6. The method according to claim 5, wherein the expected data are the values of the actual current or future physical characteristics, the comparison criterion being a distance criterion between the collected data and the expected data.

7. The method according to claim 5, wherein the expected data are the training data, the comparison criterion being a distance criterion between the collected data and the expected data.

8. A computer program product including a readable information medium, on which a computer program is recorded comprising program instructions, the computer program could be loaded on a data processing unit and adapted to cause the implementation of a method according to any one of claims 1 to 7 when the computer program is implemented on the data processing unit.

9. A readable information medium including program instructions forming a computer program, the computer program could be loaded on a data processing unit and adapted to cause the implementation of a method according to any one of claims 1 to 7 when the computer program is implemented on the data processing unit.

Images