Method for predicting current or future physical characteristics of a part of a human body and associated devices

Abstract

Method for predicting current or future physical characteristics of a part of a human body and associated devices The present invention relates to a method for predicting current or future physical characteristics of a part of a human body by a device, the part comprising skin or hair. The device is capable of acquiring images of said parts and applying a digital model thereon to predict a state of the physical characteristics of the parts of a human body, the method comprising the following phases: - first use of the device, - analysis comprising the steps of: - collecting data from the use, - determining whether the collected data satisfy a trigger criterion to obtain a determination result, - triggering an update phase based on the determination result, - updating the digital model, - second use of the device.Figure for abstract: Figure 3.

Description

Title of the invention: Method for predicting the current or future physical characteristics of a part of a human body and associated devices

[0001] The present invention relates to a method for predicting current or future physical characteristics of a part of a human body. The present invention also relates to an associated computer program product and a readable information medium.

[0002] In the world of cosmetics, it is useful to be able to offer customers products adapted to their needs quickly and precisely.

[0003] To meet this need, it is known to use devices whose purpose is to examine parts of a user's body such as hair or skin, to direct the user towards a set of products adapted to the needs of these parts.

[0004] To do this, the device implements an algorithm obtained by artificial intelligence. More precisely, the algorithm uses a set of data collected on the user and puts them as input to a digital model to determine a characteristic of the part, for example its shade. From this determined characteristic, it is then possible to give the user a suggestion of specific products.

[0005] However, in practice, it turns out that these algorithms do not have the expected precision, leading to suggestions that are not adapted to the user.

[0006] There is therefore a need for a method for predicting the current or future physical characteristics of a part of a human body which makes it possible to obtain a better prediction.

[0007] For this purpose, the present description relates to a method for predicting the current or future physical characteristics of a part of a human body by a device, the part comprising skin or hair and the device being capable of acquiring images of the part and applying a digital model to at least a part of the acquired images to obtain a predicted value of the current or future physical characteristics of a part of a human body, the digital model being obtained by machine learning, the method comprising the following phases:

[0008] - a first phase of use of the device comprising the steps of:

[0009] - acquisition of images of a user's skin or hair,

[0010] - application of the digital model to at least part of the acquired images for obtain a predicted value of the current or future physical characteristics of a part of a human body,

[0011] - an analysis phase comprising the steps of:

[0012] - collection of data from the first phase,

[0013] - determination of whether or not the collected data meet at least one criterion of trigger, to obtain a determination result,

[0014] - triggering an update phase based on the determination result,

[0015] - a phase of updating the digital model, the updating phase comprising the steps of:

[0016] - training the digital model at least on the data collected for obtain a new digital model, and

[0017] - a second phase of use of the device comprising the steps of:

[0018] - acquisition of images of the user's skin or hair,

[0019] - application of the new digital model to at least part of the images acquired to obtain a value of the current or future physical characteristics of a part of a human body.

[0020] The method thus makes it possible to obtain predicted physical characteristics with a

[0021] precision increasing with each retraining.

[0022] According to other advantageous aspects of the invention, the method comprises a or more of the following characteristics taken in isolation or in all possible combinations: - the method comprises a step of determining the distance between the collected data and the used data, the method further comprises a step of correcting the collected data when the distance is greater than or equal to a threshold value, to obtain corrected data, the training step being implemented on the corrected data.

[0023] By correcting the data before retraining, this allows for data to be obtained that allows for better convergence towards a more accurate model. - a trigger criterion is a time interval or a number of data collected. - the process includes a step of collecting user satisfaction, a trigger criterion being the percentage of customer satisfaction. - a trigger criterion is a comparison criterion between the collected data and expected data. - the expected data are the values ​​of actual current or future physical characteristics, the comparison criterion being a distance criterion between the collected data and the expected data. - the expected data are the training data, the comparison criterion being a distance criterion between the collected data and the expected data.

[0024] Each of these trigger criteria makes it possible to carry out retraining adapted to the specific case to ensure that the retraining takes place when it is really useful.

[0025] The present description relates to a computer program product comprising a readable information medium, on which is stored a computer program comprising program instructions, the computer program being loadable onto a data processing unit and adapted to cause the implementation of at least one step of a method as described previously when the computer program is implemented on the data processing unit.

[0026] The present description relates to a readable information medium comprising program instructions forming a computer program, the computer program being loadable onto a data processing unit and adapted to cause the implementation of at least one step of a method as described when the computer program is implemented on the data processing unit.

[0027] The invention will appear more clearly on reading the description which follows, given solely by way of non-limiting example and with reference to the drawings in which: - [Fig.l] [Fig.l] is a schematic view of an assembly comprising a device for predicting the current or future physical characteristics of a part of a human body, an individual using the device and a server, - [Fig.2] [Fig.2] is a detailed view of the prediction device of the [Fig.l], and - [Fig.3] [Fig.3] is a flowchart of the steps in a prediction process current or future physical characteristics of a part of a human body by the device of [Fig.2].

[0028] A prediction device 10 interacting with its environment, namely an individual 12 and a server 14 is shown in [Fig.l].

[0029] The individual 12 is generally, as is the case here, the user of the prediction device 10.

[0030] The prediction device 10 is capable of predicting current or future physical characteristics of a part of the body of the individual 12 by implementing a prediction method which will be described later.

[0031] The characteristics are features useful in determining suitable products for the individual 12, whether skin care products, makeup products or hair care products.

[0032] The characteristics are characteristics relating to the part of the body of the individual 12 or characteristics relating to the expected state of the part of the body of the individual 12 after the application of a cosmetic product.

[0033] Specific examples of characteristics are the presence or absence of wrinkles, pigmentation of the part, hydration of the part, wrinkle score representative of the degree of severity of the wrinkles, skin color, visible pores, hair diameter, percentage of white hair or hair color.

[0034] The prediction device 10 is, for example, a smartphone, a tablet or a computer.

[0035] With reference to [Fig.2], the prediction device 10 comprises an acquisition module 16, a processing module 18 and an output module 20.

[0036] The acquisition module 16 is capable of acquiring images of the individual 12.

[0037] More precisely, the acquisition module 16 acquires images of a part of the individual 12.

[0038] According to the example described, the acquisition module 16 is a camera.

[0039] In addition, the acquisition module 16 makes it possible to acquire an image in the form of a file allowing the processing of the acquired images.

[0040] The processing module 18 is capable of processing the images to predict the physical characteristics.

[0041] The operations applied to the images by the processing module 18 are digital operations which will be described later.

[0042] The processing module 18 comprises several units, namely a collection unit 22, an application unit 24, a triggering unit 26 and a correction unit 28.

[0043] These units implement the corresponding steps of the prediction method.

[0044] The output module 20 is suitable for informing the user of the results of the processing module 18.

[0045] For example, the output module 20 is a screen on which the predicted characteristic is displayed.

[0046] The prediction device 10 is in communication with the server 14.

[0047] These communications notably make it possible to carry out updates to the prediction device 10.

[0048] The operation of the prediction device 10 is now described with reference to [Fig.3] which illustrates an example of implementation of a prediction method.

[0049] The prediction method comprises a first use phase PI, an analysis phase P2, an update phase P3 and a second use phase P4.

[0050] The first phase of PI use comprises an acquisition step 100 and an application step 110.

[0051] During the acquisition step 100, the acquisition module 16 acquires images of the part of the individual 12.

[0052] During the application step 110, the processing module 18, and more specifically the application unit 24, applies a digital model to at least part of the acquired images.

[0053] The digital model takes as input at least part of the acquired images, preferably all of the acquired images, and gives as output a prediction of the current or future physical characteristics of a part of the human body of the individual 12.

[0054] The digital model used is, for example, a neural network.

[0055] Generally speaking, the neural network comprises an ordered succession of layers of neurons, each of which takes its inputs from the outputs of the previous layer.

[0056] More precisely, each layer comprises neurons taking their inputs from the outputs of the neurons of the previous layer, or from the input variables for the first layer.

[0057] Alternatively, more complex neural network structures can be envisaged with a layer that can be connected to a layer further away than the immediately preceding layer.

[0058] Each neuron is also associated with an operation, i.e. a type of processing, to be carried out by said neuron within the corresponding processing layer.

[0059] Each layer is connected to the other layers by a plurality of synapses. A synaptic weight is associated with each synapse, and each synapse forms a connection between two neurons. It is often a real number, which takes both positive and negative values. In some cases, the synaptic weight is a complex number.

[0060] Each neuron is capable of performing a weighted sum of the value(s) received from the neurons of the previous layer, each value then being multiplied by the respective synaptic weight of each synapse, or connection, between said neuron and the neurons of the previous layer, then applying an activation function, typically a non-linear function, to said weighted sum, and delivering at the output of said neuron, in particular to the neurons of the following layer which are connected to it, the value resulting from the application of the activation function. The activation function makes it possible to introduce non-linearity into the processing carried out by each neuron. Examples of activation functions are the sigmoid function, the hyperbolic tangent function, and the Heaviside function.

[0061] As an optional addition, each neuron is also capable of applying, in addition, a multiplicative factor, also called bias, to the output of the activation function, and the value delivered at the output of said neuron is then the product of the bias value and the value from the activation function.

[0062] Such a neural network is trained on a database comprising images and the prediction of each physical characteristic to be predicted.

[0063] The database is, depending on the case, more or less large, and can range from a few thousand images to a billion images.

[0064] At the end of the first phase PI, the processing module 18 has predictions for a set of images specific to the individual 12.

[0065] The second phase P2 comprises a collection step 200, a determination step 210 and a triggering step 220.

[0066] During the collection step 200, the collection unit 22 collects the data from the first phase PL

[0067] The data of the first phase PI are both the images acquired by the acquisition module 16 and the predicted values ​​for the physical characteristics.

[0068] The collection unit 22 thus collects all of this data.

[0069] For this, according to one embodiment, the collection unit 22 communicates with the processing module 18.

[0070] During the determination step 210, the triggering unit 26 determines whether or not the collected data meets at least one triggering criterion.

[0071] As will appear in the remainder of the description, the trigger criterion makes it possible to determine whether or not retraining of the digital model will be carried out.

[0072] Multiple examples can be considered for the triggering criterion(s). Some particularly advantageous examples are detailed below, knowing that these examples can be combined with each other.

[0073] According to one example, the trigger criterion is a time interval.

[0074] It can be chosen that as soon as three months have passed a retraining of the digital model will be carried out.

[0075] Another example of a trigger criterion is a number of data collected.

[0076] As an illustration, a threshold data number equal to 100,000 may be chosen. images collected.

[0077] Customer satisfaction can also be a trigger criterion.

[0078] For this, the users' opinions are collected after use of the prediction device 10, for example, in the form of a questionnaire or rating.

[0079] The results thus collected are then sent to the server 14 and an overall score will make it possible to determine whether the users are more or less satisfied with the predictions provided by the prediction device 10, and, consequently, to retrain the digital model or not.

[0080] The user may also be asked to compare the collected data with the expected data, which may be another trigger criterion. The user may be asked to see if the collected data matches their expectations via a rating. The collected data may vary depending on the environment and therefore cause errors, such as a photo taken in a place that is too bright, such as a cosmetics store.

[0081] The user will then be able to confirm whether or not the brightness is suitable and an average rating that is too low across all users will then require retraining of the digital model.

[0082] In another embodiment, the trigger criterion is a criterion for comparing the collected data with the training data of the digital model.

[0083] Typically, the distance between the collected data and the training data of the digital model will be compared with a threshold.

[0084] Probability of error or accuracy are examples of such distances.

[0085] If the distance is too high, retraining will be carried out.

[0086] During the triggering step 220, the triggering unit 26 triggers the update phase based on the result of the determination step 210, assuming here that the triggering criterion is respected.

[0087] In the event that the trigger criterion is not met, the collection unit 22 continues to collect data and implements the determination step again.

[0088] The processing module 16 thus implements the collection 200 and determination 210 steps in a loop until the trigger criterion is met.

[0089] The digital model update phase P3 aims to update the digital model to obtain a more efficient digital model.

[0090] According to the example described, the update phase P3 comprises a determination step 300, a correction step 310 and a training step 320.

[0091] During the first determination step 300, the correction unit 28 determines the distance between the collected data and the training data of the digital model.

[0092] Distance is, for example, color, brightness, resolution or customer satisfaction.

[0093] This distance can be determined digitally via calculation algorithms or by proposing a rating system to the user before the acquired image is used by the digital model.

[0094] When the distance is greater than a predetermined threshold, the correction step 310 is implemented.

[0095] During this correction step 310, the correction unit 28 corrects the collected data so that the obtained data have a distance to the training data less than the predetermined threshold.

[0096] For example, it could be chosen that each collected image must have the same average brightness across the pixels, with the brightness being changed when the image does not meet this criterion.

[0097] A set of corrected data is thus obtained.

[0098] During the training step 320, the server 14 retrains the digital model on at least the corrected data to obtain a new digital model.

[0099] This makes it possible to refine the predictions of the prediction device 10, the new model being by construction more suited to the corrected data.

[0100] In some embodiments, the retraining of the numerical model is performed on the entire data set.

[0101] The second phase of use P4 comprises an acquisition step 400 and an application step 410.

[0102] The acquisition 400 and application 410 steps are similar to the acquisition 100 and application 110 steps previously described except that the digital model used is different, the digital model being the digital model obtained at the end of the implementation of the update phase P3.

[0103] The predictions of the prediction device 10 are then more precise than the predictions obtained during the first phase of use PL

[0104] This increase in precision is obtained in a simple manner by collecting data and retraining the numerical model used.

[0105] This allows, in particular, with good robustness to adapt the digital model to the new conditions of use of the digital model.

[0106] As a specific example, a numerical model developed for a specific population is a priori not suitable for another population. With the present method, the numerical model will, over time, give predictions suitable for the other population. In this sense, the numerical model becomes a local model.

[0107] Improvements in equipment or conditions of use (sunshine) may also justify such retraining.

[0108] The method also makes it possible to resolve the problem of the unavailability of training data by gradually increasing the training base or at least by making it increasingly relevant to the desired prediction.

[0109] Other variants of the method described can be envisaged.

[0110] For example, no correction step is implemented, the retraining data being directly the collected data. In such a case, the processing module 18 does not include a correction unit 28.

[0111] According to the example described, the retraining is carried out remotely on the server 14 but it could also be envisaged to carry out the retraining locally. This could allow customization of the digital model.

[0112] Furthermore, other hardware implementations can be considered.

[0113] In the example of [Fig.2], the collection unit 22, the application unit 24 and the triggering unit 26 as well as, as an optional addition, the correction unit 28 are each produced in the form of software, or a software brick, executable by the processor. The memory of the electronic device for predicting the current or future physical characteristics of a part of a human body is then capable of storing image collection software, digital model application software and digital model retraining triggering software, as well as, as an optional addition, collected data correction software. The processor is then capable of executing each of the software among the image collection software, the digital model application software and the digital model retraining triggering software, as well as, as an optional addition, the collected data correction software.

[0114] In a variant not shown, the collection unit 22, the application unit 24 and the triggering unit 26 as well as, as an optional addition, the correction unit 28 are each produced in the form of a programmable logic component, such as an FPGA (Field Programmable Gate Array) or an integrated circuit, such as an ASIC (Application Specific Integrated Circuit).

[0115] When the electronic device for predicting the current or future physical characteristics of a part of a human body is produced in the form of one or more software programs, that is to say in the form of a computer program, also called a computer program product, it is also capable of being recorded on a medium, not shown, readable by a computer. The computer-readable medium is, for example, a medium capable of storing electronic instructions and of being coupled to a bus of a computer system. For example, the readable medium is an optical disk, a magneto-optical disk, a ROM memory, a RAM memory, any type of non-volatile memory (for example FLASH or NVRAM) or a magnetic card. A computer program comprising software instructions is then stored on the readable medium.

Claims

Claims

1. A method for predicting current or future physical characteristics of a part of a human body by a device, the part comprising skin or hair, the device being capable of acquiring images of the part, the device being capable of applying a digital model to at least a portion of the acquired images to obtain a predicted value of the current or future physical characteristics of a part of a human body, the digital model being obtained by machine learning, the method comprising the following phases: - a first phase of use of the device comprising the steps of: - acquiring images of a user's skin or hair, - applying the digital model to at least a portion of the acquired images to obtain a predicted value of the current or future physical characteristics of a part of a human body,- an analysis phase comprising the steps of: - collecting the data from the first phase, the collected data being the acquired images and the predicted value, - determining whether or not the collected data satisfy at least one trigger criterion, to obtain a determination result, - triggering an update phase based on the determination result, - a phase of updating the digital model, the update phase comprising the steps of: - training the digital model at least on the collected data to obtain a new digital model, and - a second phase of using the device comprising the steps of: - acquiring images of the user's skin or hair, - applying the new digital model to at least part of the acquired images to obtain a value of the current or future physical characteristics of a part of a human body.,

2. The method of claim 1, wherein the method further comprises: - a step of determining the distance between the collected data with training data of the digital model, - a step of correcting the collected data when the distance is greater than or equal to a threshold value, to obtain corrected data, the training step being implemented on the corrected data.

3. A method according to claim 1 or 2, wherein a trigger criterion is a time interval or a number of data collected.

4. A method according to any one of claims 1 to 3, wherein the method comprises a step of collecting user satisfaction, a trigger criterion being the customer satisfaction percentage.

5. A method according to any one of claims 1 to 4, wherein a trigger criterion is a comparison criterion between the collected data and expected data.

6. A method according to claim 5, wherein the expected data are the values ​​of actual current or future physical characteristics, the comparison criterion being a distance criterion between the collected data and the expected data.

7. The method of claim 5, wherein the expected data is the training data, the comparison criterion being a distance criterion between the collected data and the expected data.

8. Computer program product comprising a readable information medium, on which is stored a computer program comprising program instructions, the computer program being loadable onto a data processing unit and adapted to cause the implementation of the step of applying the first phase of use, the steps of the analysis phase, the step of training the update phase and the step of applying the second phase of use of a method according to any one of claims 1 to 7 when the computer program is implemented on the data processing unit.

9. A readable information carrier comprising program instructions forming a computer program, the computer program being loadable onto a data processing unit and adapted to cause the implementation of the application step of the first phase of use, of the steps of the analysis phase, of the training step of the update phase and of the application step of the second phase of use of a method according to any one of claims 1 to 7 when the computer program is implemented on the data processing unit.

Images