Method for predicting the current or future physical characteristics of a part of a human body and associated devices

Abstract

Method for predicting the current or future physical characteristics of a part of a human body and associated devices. The present invention relates to a method for predicting the current or future physical characteristics of a part of a human body by means of a device, the part comprising skin or hair. The device is capable of acquiring images of said parts and applying a digital model to them to predict a state of the physical characteristics of the parts of a human body, the method comprising the following phases: - first use of the device, - analysis comprising the steps of: - collection of data from the use, - determination whether the collected data meets a triggering criterion for obtaining a determination result, - triggering an update phase based on the determination result, - updating of the digital model, - second use of the device.Figure for the abbreviation: figure 3.

Description

Title of the invention: Method for predicting the current or future physical characteristics of a part of a human body and associated devices

[0001] The present invention relates to a method for predicting the current or future physical characteristics of a part of a human body. The present invention also relates to a computer program product and an associated readable information medium.

[0002] In the world of cosmetics, it is useful to be able to offer customers products adapted to their needs quickly and accurately.

[0003] To meet this need, it is known to use devices whose purpose is to examine parts of a user's body such as hair or skin, in order to guide the user towards a set of products adapted to the needs of these parts.

[0004] To achieve this, the device implements an algorithm derived from artificial intelligence. More specifically, the algorithm uses a set of data collected about the user and feeds it into a digital model to determine a characteristic of the part, for example, its color. Based on this determined characteristic, it is then possible to suggest specific products to the user.

[0005] However, in practice, it turns out that these algorithms do not have the expected accuracy, leading to suggestions that are not adapted to the user.

[0006] There is therefore a need for a method of predicting the current or future physical characteristics of a part of a human body which allows for a better prediction.

[0007] To this end, the present description relates to a method for predicting the current or future physical characteristics of a part of a human body by means of a device, the part comprising skin or hair and the device being capable of acquiring images of the part and applying a digital model to at least a part of the acquired images to obtain a predicted value of the current or future physical characteristics of a part of a human body, the digital model being obtained by machine learning, the method comprising the following phases:

[0008] - a first phase of use of the device comprising the steps of:

[0009] - acquisition of skin or hair images of a user,

[0010] - application of the digital model to at least a part of the images acquired for to obtain a predicted value of the current or future physical characteristics of a part of a human body,

[0011] - an analysis phase comprising the steps of:

[0012] - collection of data from the first phase,

[0013] - determination whether the collected data meet at least one criterion of triggering, to obtain a determination result,

[0014] - triggering an update phase based on the determination result,

[0015] - a phase of updating the digital model, the update phase comprising the steps of:

[0016] - training the numerical model at least on the data collected for to obtain a new digital model, and

[0017] - a second phase of use of the device comprising the steps of:

[0018] - acquisition of images of the user's skin or hair,

[0019] - application of the new digital model to at least a part of the images acquired to obtain a value of the current or future physical characteristics of a part of a human body.

[0020] The process thus makes it possible to obtain predicted physical characteristics with a

[0021] accuracy increasing with each retraining.

[0022] According to other advantageous aspects of the invention, the method comprises a or several of the following characteristics taken individually or in all possible combinations: - the process includes a step of determining the distance between the data collected with the data used, the process further includes a step of correcting the data collected when the distance is greater than or equal to a threshold value, to obtain corrected data, the training step being implemented on the corrected data.

[0023] By correcting the data before retraining, it is possible to obtain data allowing for better convergence towards a more accurate model. - a triggering criterion is a time interval or a number of data points collected. - the process includes a step of collecting user satisfaction, a trigger criterion being the percentage of customer satisfaction. - a triggering criterion is a criterion for comparing the data collected with the expected data. - the expected data are the values ​​of the actual current or future physical characteristics, the comparison criterion being a criterion of distance between the data collected and the expected data. - the expected data are the training data, the comparison criterion being a criterion of distance between the data collected and the expected data.

[0024] Each of these triggering criteria makes it possible to carry out retraining adapted to the specific case to ensure that retraining takes place when it is really useful.

[0025] The present description relates to a computer program product comprising a readable information carrier, on which is stored a computer program comprising program instructions, the computer program being loadable onto a data processing unit and adapted to cause the implementation of at least one step of a process as described above when the computer program is implemented on the data processing unit.

[0026] The present description relates to a readable information carrier comprising program instructions forming a computer program, the computer program being loadable onto a data processing unit and adapted to cause the implementation of at least one step of a process as described when the computer program is implemented on the data processing unit.

[0027] The invention will become clearer upon reading the following description, given solely by way of non-limiting example and with reference to the drawings in which: - [Fig. 1] [Fig. 1] is a schematic view of an assembly comprising a device for predicting the current or future physical characteristics of a part of a human body, an individual using the device, and a server. - [Fig.2] [Fig.2] is a detailed view of the prediction device [Fig. 1], and - [Fig. 3] [Fig. 3] is a flowchart of the steps in a prediction process current or future physical characteristics of a part of a human body by the device of [Fig.2].

[0028] A prediction device 10 interacting with its environment, namely an individual 12 and a server 14, is represented in [Fig.1].

[0029] Individual 12 is generally, as is the case here, the user of the prediction device 10.

[0030] The prediction device 10 is suitable for predicting current or future physical characteristics of a part of the body of the individual 12 by implementing a prediction process which will be described later.

[0031] The characteristics are useful features for determining the products suitable for the individual 12, whether they are skin care products, makeup products or hair care products.

[0032] The characteristics are characteristics relating to the body part of individual 12 or characteristics relating to the expected state of the body part of individual 12 after the application of a cosmetic product

[0033] Specific examples of characteristics are the presence or absence of wrinkles, pigmentation of the part, hydration of the part, wrinkle score representative of the degree of severity of wrinkles, skin color, visible pores, hair diameter, percentage of white hair or hair color.

[0034] The prediction device 10 is, for example, a smartphone, a tablet or a computer.

[0035] With reference to [Fig.2], the prediction device 10 comprises an acquisition module 16, a processing module 18 and an output module 20.

[0036] The acquisition module 16 is suitable for acquiring images of the individual 12.

[0037] More specifically, the acquisition module 16 acquires images of a part of the individual 12.

[0038] According to the example described, the acquisition module 16 is a camera.

[0039] In addition, the acquisition module 16 allows an image to be acquired in the form of a file allowing the processing of acquired images.

[0040] The processing module 18 is specific to processing images to predict physical characteristics.

[0041] The operations applied to the images by the processing module 18 are numerical operations which will be described later.

[0042] The processing module 18 comprises several units, namely a collection unit 22, an application unit 24, a triggering unit 26 and a correction unit 28.

[0043] These units implement the corresponding steps of the prediction process.

[0044] The output module 20 is designed to inform the user of the results of the processing module 18.

[0045] For example, the output module 20 is a screen on which the predicted characteristic is displayed.

[0046] The prediction device 10 is in communication with the server 14.

[0047] These communications make it possible in particular to carry out updates to the prediction device 10.

[0048] The operation of the prediction device 10 is now described with reference to [Fig.3] which illustrates an example of the implementation of a prediction method.

[0049] The prediction process comprises a first usage phase PI, an analysis phase P2, an update phase P3 and a second usage phase P4.

[0050] The first phase of PI use comprises an acquisition step 100 and an application step 110.

[0051] During acquisition step 100, the acquisition module 16 acquires images of the individual part 12.

[0052] During the application step 110, the processing module 18, and more specifically the application unit 24, applies a digital model to at least a part of the acquired images.

[0053] The digital model takes as input at least a part of the acquired images, preferably all of the acquired images, and gives as output a prediction of the current or future physical characteristics of a part of the human body of individual 12.

[0054] The digital model used is, for example, a neural network.

[0055] In general, the neural network comprises an ordered succession of layers of neurons, each of which takes its inputs from the outputs of the previous layer.

[0056] More precisely, each layer includes neurons taking their inputs from the outputs of the neurons of the previous layer, or from the input variables for the first layer.

[0057] Alternatively, more complex neural network structures can be envisaged with a layer that can be linked to a layer further away than the immediately preceding layer.

[0058] Each neuron is also associated with an operation, that is to say a type of processing, to be carried out by said neuron within the corresponding processing layer.

[0059] Each layer is connected to the other layers by a plurality of synapses. A synaptic weight is associated with each synapse, and each synapse forms a link between two neurons. It is often a real number, which takes on both positive and negative values. In some cases, the synaptic weight is a complex number.

[0060] Each neuron is designed to perform a weighted sum of the value(s) received from the neurons of the preceding layer, each value being multiplied by the respective synaptic weight of each synapse, or connection, between said neuron and the neurons of the preceding layer, and then to apply an activation function, typically a non-linear function, to said weighted sum, and to deliver at the output of said neuron, in particular to the neurons of the next layer connected to it, the value resulting from the application of the activation function. The activation function introduces non-linearity into the processing performed by each neuron. The sigmoid function, the hyperbolic tangent function, and the Heaviside function are examples of activation functions.

[0061] As an optional complement, each neuron is also capable of applying, in addition, a multiplicative factor, also called bias, to the output of the activation function, and the value delivered at the output of said neuron is then the product of the bias value and the value from the activation function.

[0062] Such a neural network is trained on a database containing images and the prediction of each physical characteristic to be predicted.

[0063] The database is, depending on the case, more or less large, possibly ranging from a few thousand images to a billion images.

[0064] At the end of the first PI phase, the processing module 18 has predictions for a set of images specific to the individual 12.

[0065] The second phase P2 comprises a collection step 200, a determination step 210 and a triggering step 220.

[0066] During collection step 200, collection unit 22 collects the data from the first phase PL

[0067] The data from the first PI phase are both the images acquired by the acquisition module 16 and the predicted values ​​for the physical characteristics.

[0068] The collection unit 22 thus collects all of this data.

[0069] To this end, according to one embodiment, the collection unit 22 communicates with the processing module 18.

[0070] During the determination step 210, the triggering unit 26 determines whether the collected data satisfies at least one triggering criterion or not.

[0071] As will appear in the rest of the description, the triggering criterion determines whether or not a retraining of the numerical model will be carried out.

[0072] Multiple examples can be considered for the triggering criterion or criteria. Some particularly advantageous examples are detailed below, bearing in mind that these examples can be combined with each other.

[0073] According to one example, the triggering criterion is a time interval.

[0074] It can be chosen that as soon as three months have passed, a retraining of the digital model will be carried out.

[0075] Another example of a triggering criterion is a number of data collected.

[0076] By way of illustration, a threshold number of data points equal to 100,000 may be chosen images collected.

[0077] Customer satisfaction can also be a triggering criterion.

[0078] For this purpose, user feedback is collected after use of the prediction device 10, for example, in the form of a questionnaire or rating.

[0079] The results thus collected are then sent to the server 14 and an overall score will determine whether the users are more or less satisfied with the predictions provided by the prediction device 10, and, consequently, whether or not to retrain the numerical model.

[0080] The user may also be asked to compare the collected data with the expected data, which can be another triggering criterion. The user may be asked to rate whether the collected data meets their expectations. The collected data may vary depending on the environment and therefore lead to errors, such as a photo taken in an overly bright location like a cosmetics store.

[0081] The user will then be able to confirm whether or not the brightness is suitable and an average rating that is too low across all users will then imply a retraining of the digital model.

[0082] In another embodiment, the triggering criterion is a criterion for comparing the collected data with the training data of the numerical model.

[0083] Typically, the distance between the collected data and the training data of the numerical model will be compared with a threshold.

[0084] The probability of error or the precision are examples of such distances.

[0085] If the distance is too great, retraining will be carried out.

[0086] During the triggering step 220, the triggering unit 26 triggers the update phase based on the result of the determination step 210, assuming here that the triggering criterion is met.

[0087] In the event that the triggering criterion is not met, the collection unit 22 continues to collect data and implements the determination step again.

[0088] The processing module 16 thus implements in a loop the collection 200 and determination 210 steps until the triggering criterion is met.

[0089] The P3 numerical model update phase aims to update the numerical model to obtain a more efficient numerical model.

[0090] According to the example described, the update phase P3 comprises a determination step 300, a correction step 310 and a training step 320.

[0091] During the first determination step 300, the correction unit 28 determines the distance between the collected data and the training data of the numerical model.

[0092] Distance is, for example, colour, brightness, resolution or customer satisfaction.

[0093] This distance can be determined numerically via calculation algorithms or by proposing a notation system to the user before the acquired image is used by the digital model.

[0094] When the distance is greater than a predetermined threshold, the correction step 310 is implemented.

[0095] During this correction step 310, the correction unit 28 corrects the collected data so that the data obtained has a distance from the training data that is less than the predetermined threshold.

[0096] For example, it could be chosen that each collected image must have the same average brightness on the pixels, with the brightness being changed when the image does not meet this criterion.

[0097] A corrected dataset is thus obtained.

[0098] During the training step 320, the server 14 performs a retraining of the numerical model on at least the corrected data to obtain a new numerical model.

[0099] This allows the predictions of the prediction device 10 to be refined, the new model being by construction more suited to the corrected data.

[0100] In some embodiments, the retraining of the numerical model is carried out on the entire dataset.

[0101] The second phase of use P4 comprises an acquisition step 400 and an application step 410.

[0102] The acquisition steps 400 and application steps 410 are similar to the acquisition steps 100 and application steps 110 previously described except that the digital model used is different, the digital model being the digital model obtained at the end of the implementation of the update phase P3.

[0103] The predictions of the prediction device 10 are then more accurate than the predictions obtained during the first phase of use PL

[0104] This increase in accuracy is obtained simply by collecting data and retraining the numerical model used.

[0105] This allows, in particular, with good robustness, the digital model to be adapted to new conditions of use of the digital model.

[0106] As a specific example, a numerical model developed for a specific population is not necessarily suitable for another population. With the present method, the numerical model will gradually provide predictions adapted to the other population. In this sense, the numerical model becomes a local model.

[0107] Improvements in equipment or conditions of use (sunlight) may also justify such retraining.

[0108] The process also makes it possible to solve the problem of the unavailability of training data by increasing the training base over time or at least by making it more and more relevant to the desired prediction.

[0109] Other variations of the described process may be envisaged.

[0110] For example, no correction step is implemented, the retraining data being directly the collected data. In such a case, the processing module 18 does not include a correction unit 28.

[0111] According to the example described, retraining is performed remotely on server 14, but it could also be considered to perform retraining locally. This could allow for customization of the digital model.

[0112] Furthermore, other hardware implementations may be envisaged.

[0113] In the example of [Fig. 2], the data acquisition unit 22, the application unit 24, and the triggering unit 26, as well as the optional correction unit 28, are each implemented as software, or a software component, executable by the processor. The memory of the electronic device for predicting the current or future physical characteristics of a part of a human body is thus capable of storing image acquisition software, software for applying a digital model, and software for triggering the retraining of a digital model, as well as, optionally, software for correcting the collected data. The processor is then capable of executing each of the following software programs: image acquisition software, digital model application software, and digital model retraining triggering software, as well as, optionally, software for correcting the collected data.

[0114] In an alternative not shown, the collection unit 22, the application unit 24 and the triggering unit 26, as well as the optional correction unit 28, are each implemented as a programmable logic component, such as an FPGA (Field Programmable Gate Array), or as an integrated circuit, such as an ASIC (Application Specified Integrated Circuit).

[0115] When the electronic device for predicting the current or future physical characteristics of a part of a human body is implemented in the form of one or more software programs, i.e., in the form of a computer program, also called a computer program product, it is further capable of being stored on a computer-readable medium, not shown. The computer-readable medium is, for example, a medium capable of storing electronic instructions and being connected to a bus of a computer system. By way of example, the readable medium is an optical disc, a magneto-optical disc, ROM, RAM, any type of non-volatile memory (e.g., FLASH or NVRAM), or a magnetic card. A computer program containing software instructions is then stored on the readable medium.

Claims

Demands

1. A method for predicting the current or future physical characteristics of a part of a human body by means of a device, the part comprising skin or hair, the device being adapted to acquire images of the part, the device being adapted to apply a digital model to at least a portion of the acquired images to obtain a predicted value of the current or future physical characteristics of a part of a human body, the digital model being obtained by machine learning, the method comprising the following phases: - a first phase of using the device comprising the steps of: - acquiring images of a user's skin or hair, - applying the digital model to at least a portion of the acquired images to obtain a predicted value of the current or future physical characteristics of a part of a human body,- an analysis phase comprising the following steps: - data collection from the first phase, the collected data being the acquired images and the predicted value, - determination of whether the collected data meets at least one triggering criterion, to obtain a determination result, - triggering an update phase based on the determination result, - a digital model update phase, the update phase comprising the following steps: - training the digital model on at least the collected data to obtain a new digital model, and - a second phase of device use comprising the following steps: - acquisition of skin or hair images of the user, - application of the new digital model to at least a portion of the acquired images to obtain a value for the current or future physical characteristics of a part of a human body.

2. A method according to claim 1, wherein the method further comprises: - a step of determining the distance between the data collected with training data of the numerical model, - a step of correcting the data collected when the distance is greater than or equal to a threshold value, to obtain corrected data, the training step being implemented on the corrected data.

3. A method according to claim 1 or 2, wherein a triggering criterion is a time interval or a number of data points collected.

4. A method according to any one of claims 1 to 3, wherein the method includes a step of collecting user satisfaction, a triggering criterion being the percentage of customer satisfaction.

5. A method according to any one of claims 1 to 4, wherein a trigger criterion is a comparison criterion between the data collected and the expected data.

6. A method according to claim 5, wherein the expected data are the values ​​of the actual current or future physical characteristics, the comparison criterion being a distance criterion between the collected data and the expected data.

7. A method according to claim 5, wherein the expected data are the training data, the comparison criterion being a distance criterion between the collected data and the expected data.

8. Product computer program comprising a readable information carrier, on which is stored a computer program comprising program instructions, the computer program being loadable onto a data processing unit and adapted to drive the implementation of the application step of the first phase of use, the steps of the analysis phase, the training step of the update phase and the application step of the second phase of use of a method according to any one of claims 1 to 7 when the computer program is implemented on the data processing unit.

9. A readable information carrier containing program instructions forming a computer program, the computer program being loadable onto a data processing unit and adapted to drive the implementation of the application step of the first phase of use, the steps of the analysis phase, the training step of the update phase and the application step of the second phase of use of a method according to any one of claims 1 to 7 when the computer program is implemented on the data processing unit.

Images