Generating training datasets for training machine learning algorithms

Abstract

A mechanism for modifying a training dataset usable to train a machine learning algorithm. A trained machine learning algorithm is used to perform a desired task on an input image generated by a camera of a personal care device. In one embodiment, a training input image of a training dataset for a machine learning algorithm is modified in response to one or more values of one or more device characteristics of a personal care device.

Description

Technical Field

[0001] The present invention relates to the field of machine learning algorithms, particularly those for processing images generated by personal care devices. Background Art

[0002] It is well known that there is an increasing trend to integrate cameras (i.e., or imaging sensors) into personal care devices. Images generated by such cameras can be processed, for example, using machine learning algorithms to produce desired output data. The desired output data can be used, for example, for remote image-based diagnosis, position sensing, therapy planning, or process monitoring.

[0003] Therefore, it is quite common to process one or more images generated by a camera of a personal care device using machine learning algorithms to perform desired tasks on the (multiple) images.

[0004] Examples of personal care devices are well known to those skilled in the art and can include any device that can be used to improve personal hygiene, enhance personal appearance, and / or assist in personal body functions. Suitable examples include: toothbrushes, oral appliances, razors, hair trimmers, hair removal devices, oral irrigators, intense pulsed light (IPL) treatment devices for hair removal, breast pumps, etc. Summary of the Invention

[0005] The present invention is defined by the claims.

[0006] According to an example aspect of the present invention, there is provided a computer-implemented method for modifying a training data set that can be used to train a machine learning algorithm to perform a desired task on input images generated by a camera of a personal care device.

[0007] The computer-implemented method includes: for each device characteristic among one or more device characteristics of a personal care device, obtaining the value of the device characteristic; obtaining a training data set for the machine learning algorithm, the training data set including a plurality of training data entries, each training data entry including a training input image and training output data, the training output data representing data generated by performing a desired task on the training input image; and using the value of each device characteristic to modify each training input image to produce a modified training data set.

[0008] The object of the present disclosure is to be able to perform robust analysis or processing (e.g., classification or noise reduction) on images acquired with a camera integrated in a personal care device.

[0009] It has been recognized that conventional training input images (for training data) for training machine learning algorithms to perform desired tasks are typically obtained in a different manner from the settings in (clinical) practice. For example, such training input images can be generated using professional standard cameras or imaging devices. Thus, the images on which the machine learning algorithms are trained will not fully represent the images obtained with the camera of a personal care device.

[0010] For example, the inherent motion of a personal care device (e.g., vibration, rotation, or translation) will result in blurred / distorted images. Similarly, the characteristics of the camera will also affect the image quality (e.g., due to resolution issues), and if there is also motion, the image quality will be particularly enhanced. This blur / distortion will significantly affect the performance of the machine learning algorithm.

[0011] To overcome this problem, this paper proposes using device characteristics specific to the personal care device parameters, i.e., before performing the training of the machine learning algorithm, adding / modifying the training input images. This approach means that the training input images are processed such that before they are fed into the algorithm training, the specific blur / distortion effects (of a specific personal care device) are artificially incorporated. In this way, it is possible to train (modified) machine learning algorithms for each individual personal care device or a certain type / model of personal care device.

[0012] This improves the accuracy of the algorithm by making the subsequently trained machine learning algorithm more specific to a particular personal care device.

[0013] The value of at least one device characteristic can be generated by one or more sensors of the personal care device.

[0014] In some embodiments, at least one of the one or more device characteristics responds to or indicates the motion of the personal care device during use. It has been recognized that the motion of the personal care device will cause the images generated by the camera of the personal care device to be blurred / distorted. This blur / distortion can be simulated in the training input images to improve the training dataset for training the machine learning algorithm.

[0015] The one or more device characteristics can include one or more of the following: the vibration frequency of the personal care device and / or the vibration element of the personal care device during use; the vibration amplitude of the personal care device and / or the vibration element of the personal care device during use; the vibration waveform or pattern of the personal care device and / or the vibration element of the personal care device during use (e.g., in-plane sweep, reciprocating vibration, out-of-plane tapping); the rotation frequency of the rotating element of the personal care device during use; and / or the rotation waveform or pattern of the rotating element of the personal care device during use (e.g., rotation-oscillation).

[0016] In some examples, at least one of the one or more device characteristics is a characteristic of a camera of a personal care device. It is recognized that the attributes of the camera of the personal care device can be different from those of the camera of the captured training input images. Therefore, the training input images can be modified such that they more closely resemble equivalent images captured by the camera of the personal care device, thereby improving the relevance of the training dataset for processing such images.

[0017] The one or more device characteristics can include one or more of the following: the frame rate of the camera; an indicator of whether the camera uses global shutter technology; an indicator of whether the camera uses rolling shutter technology; if the camera uses rolling shutter technology, the line collection time and / or the time delay between the exposures of adjacent image lines; if the camera uses global shutter technology, the duty cycle of image acquisition; and / or one or more lighting parameters of the camera.

[0018] The method can further include the step of obtaining the value of each user characteristic of one or more user characteristics of a user of the personal care device. The step of modifying each training input image can similarly include using the value of each device characteristic and each user characteristic to modify each training input image to produce a modified training dataset.

[0019] The method can further include the step of using the modified training dataset to train a machine learning algorithm.

[0020] The step of training the machine learning algorithm can include iteratively performing the following steps: processing each training input image using the machine learning algorithm to produce predicted output data for each training input image; using a loss function to determine the error between the predicted output data and the training output data; and modifying the machine learning algorithm in response to the determined error. Suitable loss functions and other methods for determining the error are well known in the art.

[0021] A computer-implemented method for performing a desired task on an input image generated by a camera of a personal care device is also proposed, the computer-implemented method including: obtaining the input image; and processing the input image using a machine learning algorithm that is trained using any of the methods described herein to perform the desired task on the input image.

[0022] A training dataset is also provided, which can be obtained by performing the methods described herein for modifying a training dataset that can be used to train a machine learning algorithm to perform a desired task on an input image generated by a camera of a personal care device.

[0023] There is also provided a non-transitory storage medium that stores a training data set that can be obtained by performing the methods described herein, which are used to modify a training data set that can be used to train a machine learning algorithm to perform a desired task on input images generated by a camera of a personal care device.

[0024] Preferably, any of the methods described herein is performed by a processing system independent of the personal care device, such as a cloud computing processing system.

[0025] There is also provided a computer program product that includes computer program code portions that, when executed by a processing system, cause the processing system to perform all of the steps of any of the methods described herein.

[0026] There is also provided a processing system that is used to modify a training data set that can be used to train a machine learning algorithm to perform a desired task on input images generated by a camera of a personal care device, and the processing system is configured to: for each device characteristic among one or more device characteristics of the personal care device, obtain the value of the device characteristic; obtain a training data set for the machine learning algorithm, the training data set including a plurality of training data entries, each training data entry including a training input image and training output data, the training output data representing data generated by performing the desired task on the training input image; and use the value of each device characteristic to modify each training input image to generate a modified training data set.

[0027] The value of each device characteristic can be generated by one or more sensors of the personal care device.

[0028] Preferably, at least one of the one or more device characteristics responds to or indicates the movement of the personal care device during use.

[0029] In some examples, at least one of the one or more device characteristics is a characteristic of the camera of the personal care device.

[0030] With reference to the (multiple) embodiments described below, these and other aspects of the present invention will become clear and be elucidated. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] To better understand the present invention and more clearly show how to implement the present invention, reference will now be made, by way of example only, to the accompanying drawings, in which:

[0032] Figure 1 A method according to one embodiment is illustrated;

[0033] Figure 2 The sway effect in the input image is illustrated; and

[0034] Figure 3 Illustrated is a processing system according to one embodiment. Detailed Description

[0035] The present invention will be described with reference to the accompanying drawings.

[0036] It should be understood that the detailed description and specific examples, although indicating exemplary embodiments of the apparatus, systems and methods, are for illustrative purposes only and are not intended to limit the scope of the invention. These and other features, aspects and advantages of the apparatus, systems and methods of the present invention will become better understood from the following description, the appended claims and the drawings. It should be understood that the drawings are merely schematic and are not drawn to scale. It should also be understood that in all the drawings, the same reference numerals are used to indicate the same or similar parts.

[0037] The present invention provides a mechanism for modifying a training data set that can be used to train a machine learning algorithm. The trained machine learning algorithm is used to perform a desired task on an input image generated by a camera of a personal care device. In response to one or more values of one or more device characteristics of the personal care device, the training input images of the training data set for the machine learning algorithm are modified.

[0038] The present disclosure relates to a process for modifying or adjusting a training data set that is used to train a machine learning algorithm, particularly a machine learning algorithm configured to perform a desired task on an input image provided by a camera of a personal care device.

[0039] Those skilled in the art will readily understand that various different desired tasks for processing input images will be clear to those skilled in the art.

[0040] The desired task is preferably an image (i.e., digital image) enhancement and / or analysis technique or task, i.e., a task that includes digital enhancement or analysis performance of the input image. Example tasks may include denoising, object detection, image classification, image segmentation, image quality estimation, and the like.

[0041] The present disclosure provides a method for improving the performance of such machine learning algorithms in a specific use case scenario of a personal care device. The nature and use of the personal care device means that there are special difficulties in providing a general machine learning algorithm to process images captured by a camera of such a personal care device. The present invention provides a method for overcoming this problem.

[0042] It is well known that a machine learning algorithm is any self-training algorithm that processes input data in order to produce or predict output data. Here, the input data includes the input images provided by the camera of the personal care device, and the output data includes the results of the desired tasks performed on the input images.

[0043] The example output data can for example include a denoised input image, a classification of the input image, a location identification of the representation of a desired object within the input image, and a segmentation result of the input image. Other examples will be clear to a person skilled in the art, and the present invention is not limited to specific image enhancement, analysis, or other image processing techniques performed by the desired task.

[0044] Suitable machine learning algorithms for the present invention will be clear to a person skilled in the art. Examples of suitable machine learning algorithms include decision tree algorithms and artificial neural networks. Other machine learning algorithms such as logistic regression, support vector machines, or naive Bayes models are suitable alternatives.

[0045] The structure of an artificial neural network (or simply a neural network) is inspired by the human brain. A neural network consists of layers, each layer including a plurality of neurons. Each neuron includes mathematical operations. In particular, each neuron can include different weighted combinations of a single type of transformation (e.g., the same type of transformation, sigmoid, etc., but with different weights). During the process of processing input data, each neuron performs mathematical operations on the input data to produce a numerical output, and the output of each layer in the neural network is successively fed into the next layer. The last layer provides the output.

[0046] Methods for training machine learning algorithms are well known. Typically, such methods include obtaining a training data set, which includes training data entries. Each training data entry includes a training input data entry and a training output data entry. An initialized machine learning algorithm is applied to each training input data entry to generate a predicted output data entry. The error between the predicted output data entry and the corresponding training output data entry is used to modify the machine learning algorithm. Typically, the error is determined using a loss function (sometimes referred to as a cost function). This process can be repeated until the error converges and the predicted output data entry is sufficiently similar to the training output data entry (e.g., ±1%). This is often referred to as supervised learning technology.

[0047] For example, in the case where the machine learning algorithm is formed by a neural network, the (weights) of the mathematical operations of each neuron can be modified until the error converges. Known methods for modifying neural networks include gradient descent, backpropagation algorithms, etc.

[0048] The training input data entries correspond to example input images (i.e., include training input images). Thus, each training input data entry is a training input image. The training output data entries correspond to example results of the desired task performed on the input images. Thus, each training output data entry is training output data.

[0049] As previously mentioned, the present invention relates to an improvement of a training dataset that can be used to train a machine learning algorithm for processing input images provided by a personal care device. This in turn results in an improved machine learning algorithm.

[0050] It has been recognized that in current practice, training datasets for training machine learning algorithms are generated in a controlled environment (such as a clinical environment) and / or using professional imaging devices and / or by trained / skilled operators. As such, the images included in the training dataset do not represent the images actually captured by a personal care device.

[0051] By way of example, the movement of a personal care product or its components during use may cause image distortion / blurring, while there is no movement in a controlled environment. This movement can be generated, for example, by the vibration or rotational movement of a personal care device or its components. It is readily understood that a variety of different personal care devices may have moving components, such as an electric toothbrush, a mouthpiece, a razor, a hair trimmer, a breast pump (at least caused by the pulsating movement due to suction backpressure) and / or an oral irrigator.

[0052] As another example, the properties of the camera of a personal care device are unlikely to be the same as those of the camera used to capture the images in the training dataset. This will result in different types / structures of images being generated by the camera of the personal care device compared to the images in the training dataset.

[0053] As another example, some personal care devices may introduce or provide additional light sources that do not exist when generating the training dataset. For example, an intense pulsed light (IPL) hair removal device generates a flash of light, which affects the exposure of any images captured by the personal care device.

[0054] Of course, a simple way to solve this problem is to use a training dataset that includes images captured by the camera of a personal care device as its input data entries, and the images have been appropriately processed to perform the desired task (such as classification or denoising). However, this method creates a significant burden when creating the necessary output data entries for performing precise training. For example, if the desired task performed by the machine learning algorithm is a classification task, experts will need a large amount of time to examine and label the images generated by each personal care device in order to generate a good machine learning algorithm for that personal care device.

[0055] The present disclosure proposes an alternative method to overcome the above problems. In particular, the present disclosure proposes a method for modifying or adjusting an existing training dataset to reflect the specific characteristics of a personal care device.

[0056] In particular, it has been recognized that the exact degree of blurriness / distortion (or other characteristics) of any image captured by a camera of a personal care device depends on device-specific parameters of the individual device, including characteristics of the device itself and / or the device user.

[0057] The present disclosure presents a method of applying device-specific enhancements to images in a training data set that, when used to train a machine learning algorithm, results in a machine learning algorithm that is robust to specific image attributes (e.g., caused by specific blurriness and / or distortion) caused by using a specific personal care device.

[0058] Figure 1 FIG. 100 is a flow chart of a method or process 100 employed in an embodiment of the present invention. Process 100 is used to modify a training data set that can be used to train a machine learning algorithm to perform a desired task on an input image generated by a camera of a personal care device.

[0059] Process 100 includes a step 110 of obtaining a value of each device characteristic among one or more device characteristics for a personal care device. Each device characteristic is preferably a characteristic that, if changed, would affect the distortion and / or blurriness (or other image characteristics) of any image captured by the camera of the personal care device. More detailed examples are given later in the present disclosure.

[0060] For example, the (multiple) value(s) of the (multiple) device characteristic(s) can be obtained from the personal care device itself. Preferably, one or more sensors of the personal care device are configured to generate at least one of the (multiple) values. In some examples, at least one of the (multiple) values is stored in a memory of the personal care device. In some embodiments, at least one of the values is stored in another memory (e.g., a cloud-based memory system).

[0061] Process 100 further includes a step 120 of obtaining a training data set for a machine learning algorithm. The training data set includes a plurality of training data entries, each training data entry including a training input image and training output data that represents data generated by performing a desired task on the training input image.

[0062] In step 120, the training data set can be obtained from a database or other form of memory storing the training data set. For example, this can be a cloud-based database or memory.

[0063] Process 100 further includes step 130 of modifying each training input image using the value of each device characteristic to produce a modified training data set. Thus, the training input images of each training data entry are modified using the value of each device characteristic. The training output data of the training data entry remains unchanged. This produces a new training data set with modified data. In particular, the new training data set is personalized or customized for a specific personal care device.

[0064] Thus, step 130 is actually a step of augmenting the training data set using the characteristics of the personal care device. This adapts the training data set to the specific inherent properties or uses of the personal care device.

[0065] Step 130 may include, for example, processing the value(s) of the device characteristic(s) to determine or predict the effect of the value(s) on the image generated by the camera of the personal care device. Step 130 may then perform a process of modifying each training input image to simulate or emulate the same effect in each training input image.

[0066] It should be understood that different values of the device characteristic will affect the image generated by the camera of the personal care device to different degrees. Thus, the training input images may be adapted to simulate or emulate a similar effect.

[0067] Process 100 may further include step 140 of training a machine learning algorithm using the modified training data set. The method of training a machine learning algorithm using a training data set has been well established and has been described previously.

[0068] By way of example, step 140 may include iteratively performing the following steps: processing each training input image using the machine learning algorithm to produce predicted output data for each training input image; using a loss function to determine the error between the predicted output data and the training output data; and modifying the machine learning algorithm in response to the determined error.

[0069] The iteration may be performed until the error converges, until a predetermined number of iterations have been performed, and / or until the error value drops below a certain predetermined value. Other suitable criteria for terminating the iteration will be clear to those skilled in the art.

[0070] Naturally, the modification performed in step 130 will depend at least in part on the device characteristic(s) for which one or more values are obtained in step 110.

[0071] Preferably, the value(s) of at least one of the (multiple) device characteristics is / are generated by one or more sensors of the personal care device. The use of one or more sensors provides device- or usage-specific information for augmenting the training data set, thus providing a more accurate and usage-specific machine learning algorithm.

[0072] In a preferred embodiment, at least one of the one or more device characteristics responds to or indicates the movement of the personal care device during use. It has been recognized that such characteristics will significantly affect the blur / distortion of any image captured by the camera of the personal care device. Examples of suitable sensors that can be used to capture such device characteristics include an inertial measurement unit (IMU), an accelerometer, and / or a gyroscope.

[0073] Multiple examples of how to use the value(s) of the (multiple) device characteristics to modify the input images of the training data set are described below. It should be understood that these provide a non-exhaustive range of examples that help to illustrate how to use the proposed method to improve the training data set that can be used to train a machine learning algorithm.

[0074] Any combination of one or more of the following example methods can be used.

[0075] In a first example, one or more device characteristics include the image resolution of an image captured by the camera of the personal care device. This information can be stored, for example, in the memory of the personal care device or other storage unit. Alternatively, this information can be obtained by directly analyzing the image captured by the camera of the personal care device. Alternatively, this information can be stored in an external database, for example, the external database can be looked up using the device identification information (such as the device model identifier) of the personal care device.

[0076] In this first example, step 130 of modifying the training input images in the training data set can include adjusting the image resolution of the training input images to match the resolution of the image captured by the camera of the personal care device. This can include, for example, performing a downsampling process (to reduce the resolution) on each training input image or performing a super-resolution process (to increase the resolution) on each training input image.

[0077] In a second example, one or more device characteristics include the field of view of the camera of the personal care device. The field of view defines the amount of the world that can be observed by the camera of the personal care device, for example, it can be defined by the viewing angle and / or focal length, etc. This information can be stored, for example, in the memory of the personal care device or other storage unit. Alternatively, this information can be stored in an external database, for example, the external database can be looked up using the device identification information (such as the device model identifier) of the personal care device.

[0078] In this second example, step 130 of modifying the training input images in the training dataset may include adjusting the field of view of each training input image to match the field of view of any image captured by the camera of the personal care device. In particular, the field of view of each training input image may be known (e.g., based on the characteristics of the camera(s) that captured the training input image(s)). The field of view may then be modified, e.g., the training input images may be appropriately cropped and / or blurred to match the (expected) field of view of the images captured by the camera of the personal care device.

[0079] In a third example, the camera of the personal care device uses a rolling shutter technique to capture images. In such a scenario, it should be understood that the movement of the personal care device (e.g., due to a vibrating element of the personal care device, such as a toothbrush) will cause a "wobbling" effect to appear in the captured images because the position of the camera may change slightly between the capture of different rows (causing a perceived pixel shift between adjacent rows or lines of the captured image).

[0080] This effect is illustrated in Figure 2 which Figure 2 shows blur / distortion in the intraoral image 200 recorded with a vibrating oral care device (e.g., an electric toothbrush). The wobbling effect can clearly be seen as distinct wavy lines in the vertical direction. This is the result of different amounts of horizontal pixel shift (caused by the vibration) between the capture of different lines of the intraoral image.

[0081] In this third example, one or more device characteristics may include: an indicator of whether the camera uses a rolling shutter technique; the time delay between the exposures of adjacent image rows; the vibration frequency of the vibrating element and the vibration amplitude of the vibrating element.

[0082] The vibration frequency and / or amplitude may be determined using, for example, a vibration sensor (such as an accelerometer) carried by the personal care device. A vibration sensor is any sensor that generates a signal in response to an amount of vibration and is designed to determine the amount of vibration. As another example, this information may be stored in the memory or other storage unit of the personal care device or an external memory device (e.g., retrievable using the device identification information of the personal care device (such as a device model identifier)).

[0083] The absolute vibration amplitude (e.g., in real space) may be used in conjunction with information about the camera to derive the corresponding maximum pixel shift amount within the image captured by the camera. For example, the relationship between the absolute vibration amplitude and the pixel size may be used to predict or determine the maximum possible pixel shift amount of the image due to vibration.

[0084] The values of these device characteristics can be used to simulate the same "wobbling" effect in the training input images. In particular, the time delay and the vibration frequency can be used to predict the pixel shift between adjacent rows, for example, as a percentage of the maximum possible pixel shift amount. The vibration amplitude can be used to predict the maximum pixel shift amount of an image data row. In particular, if the time delay and the vibration frequency are known, it is possible to identify which rows of the training input image should be shifted to simulate the wobbling effect, and to what extent each row should be shifted (e.g., up to the maximum pixel shift amount).

[0085] Thus, in this third example, the step 130 of modifying the training input images in the training data set can include simulating the wobbling effect caused by the personal care device.

[0086] In a fourth example, a camera of a personal care device uses a global shutter technique to capture images. In such a scenario, it should be understood that the movement of the personal care device (e.g., due to a vibration element of the personal care device, such as a toothbrush) can cause motion blur in the images captured by the camera of the personal care device.

[0087] In this fourth example, one or more device characteristics can at least include the vibration amplitude of the personal care device. In step 130, the vibration amplitude can be used to simulate motion blur in the training input images.

[0088] The vibration amplitude can be determined using, for example, a vibration sensor (such as an accelerometer) carried by the personal care device. As another example, this information can be stored in a memory or other storage unit of the personal care device or an external memory device (e.g., retrievable using the device identification information (such as a device model identifier) of the personal care device).

[0089] In a more advanced embodiment of the fourth example, one or more device characteristics can also include the duty cycle of image acquisition and the vibration frequency. This can be used to predict the total amount of vibration that occurs to affect the motion blur in the images captured by the camera of the personal care device, which can be simulated in step 130.

[0090] This information can be stored in a memory or other storage unit of the personal care device or an external memory device (e.g., retrievable using the device identification information (such as a device model identifier) of the personal care device). In some examples, the vibration frequency and / or amplitude can be determined using a vibration sensor carried by the personal care device.

[0091] In a fifth example, one or more device characteristics may include one or more illumination characteristics of a camera of a personal care device. Different illumination characteristics will generate images with different illumination attributes (e.g., different brightness, contrast, color / wavelength type, etc.). The illumination may be caused, for example, by the nature of the camera itself or by another light source, such as a light source carried by the personal care device, such as a laser for performing IPL.

[0092] This information can be used to define or modify the illumination or illumination characteristics of the training input images. For example, the brightness, color, and / or contrast of each training input image can be modified in response to the value of the illumination characteristic.

[0093] More particularly, the illumination characteristics can be used to define or predict the expected (average) brightness, contrast, and / or color of an image captured by a camera of a personal care device. This information can be used to modify the corresponding attributes of the training (multiple) input images. This more closely aligns the illumination attributes of the (multiple) training input images with the illumination attributes of the camera of the personal care device.

[0094] This information can be stored in a memory or other storage unit of the personal care device or an external memory device (e.g., retrievable using the device identification information of the personal care device, such as the device model identifier).

[0095] In a sixth example, a rolling shutter technique is used by a camera of a personal care device to capture images. In such a scenario, it should be understood that pulsed light emitted by a light source carried by the personal care device will result in different illumination characteristics for different line captures.

[0096] In this sixth example, one or more device characteristics may include: an indicator of whether the camera uses a rolling shutter technique; the time delay between the exposures of adjacent image lines; the pulse intensity, pulse frequency, and / or pulse duration of the flash of a light source carried by the personal care device.

[0097] For each line capture of an image generated by a camera of a personal care device, this information can be used to predict the expected brightness, color, and / or contrast (or any other illumination characteristic). This information can then be used to appropriately modify the corresponding lines of each training input image to match the expected brightness, color, and / or contrast (e.g., if the camera uses a rolling shutter technique).

[0098] Any combination of the above examples can be used to modify the training input images.

[0099] The above examples provide a range of various methods for using different examples of device characteristics to modify training input images.

[0100] A person skilled in the art will readily understand other methods for modifying the training input images based on the device characteristics of the personal care device such that the characteristics of the training input images closely match or align with the characteristics of the images captured by the camera of the personal care device.

[0101] Suitable example device characteristics include: the vibration frequency of the personal care device and / or the vibration element of the personal care device during use; the vibration amplitude of the personal care device and / or the vibration element of the personal care device during use; and / or the rotation frequency of the rotating element of the personal care device during use.

[0102] Other suitable example device characteristics include: the vibration waveform or pattern of the personal care device and / or the vibration element of the personal care device during use; and / or the rotation waveform or pattern of the rotating element of the personal care device.

[0103] Knowing the exact vibration or rotation waveform will allow for a more accurate identification of the impact of the vibration / rotation on the images generated by the camera of the personal care device. For example, this can allow for a more precise modification of the training input images to simulate the same effect.

[0104] However, other suitable device characteristics include the frame rate of the camera; an indicator of whether the camera uses global shutter technology; an indicator of whether the camera uses rolling shutter technology; if the camera uses rolling shutter technology, the row collection time and / or the time delay between the exposures of adjacent image rows; if the camera uses global shutter technology, the duty cycle of image acquisition; and / or one or more lighting parameters of the camera.

[0105] In some embodiments, step 120 may be adapted to include obtaining a training data set in response to the value(s) of one or more device characteristics.

[0106] In particular, step 120 may include selecting a subset of training data entries from a larger pool of training data entries to form the training data set. The subset will not include the entire pool of training data entries. For example, the training data entries in the subset may be selected to include training input images that are similar or have similar characteristics to the images that will be generated by the personal care device in use.

[0107] By way of example, each training data entry in the pool of training data entries may be associated with a value of each of one or more potential device characteristics. Step 120 may include selecting those values that match or correspond to the value(s) of the device characteristic(s) obtained in step 110.

[0108] By way of pure working examples, each training data entry can include an indicator that indicates whether the training input image (of that training data entry) was acquired using a rolling shutter technique or a global shutter technique. The (multiple) device characteristics of the personal care device can include an indicator regarding whether the camera of the personal care device uses a rolling shutter or a global shutter technique to capture images. The selection of a subset of the training data entries can include selecting (only) those entries captured using the same shutter technique as the camera of the personal care device.

[0109] As another example, each training data entry can include an indicator of the field of view of the training input image. The (multiple) device characteristics of the personal care device can include information regarding the field of view of the camera of the personal care device. The selection of a subset of the training data entries can include (only) selecting those entries within a predetermined range of the field of view (e.g., ±10%).

[0110] Process 100 can also include step 115 of obtaining the value of each user characteristic among one or more user characteristics of the user of the personal care device. Step 130 of modifying each training input image includes using the values of each device characteristic and each user characteristic to modify each training input image to produce a modified training data set.

[0111] Example user characteristics can include, for example, the motion pattern of the personal care device of the user of the personal care device. The motion pattern can, for example, represent the movement of the user with respect to the personal care device, which affects the images captured by the camera of the personal care device. The modification of the (multiple) training input images of the training data set can, for example, include simulating the effects of such movement (e.g., motion blur, etc.).

[0112] Figure 3 Illustrated is a processing system 300 configured to perform the previously described method 100.

[0113] Processing system 300 is independent of the personal care device, e.g., not physically connected to personal care device 350. The processing system can, for example, include a cloud computing system 310 and / or a mobile / cellular device 320 (such as a mobile / cellular phone, smartphone, laptop computer, or tablet computer).

[0114] Processing system 300 is configured to obtain the value of each device characteristic among one or more device characteristics of the personal care device.

[0115] The processing system can obtain at least some of the (multiple) values from the personal care device. Thus, processing system 300 can be configured to communicate with the personal care device (e.g., via a wireless communication channel) to retrieve or receive the (multiple) values of the (multiple) device characteristics.

[0116] The processing system 300 is also configured to obtain a training data set for a machine learning algorithm. The training data set can be stored, for example, by a separate memory 340 or a memory system (e.g., a cloud computing memory system). As previously described, the training data set includes a plurality of training data entries, each training data entry including a training input image and training output data representing the data generated by performing a desired task on the training input image.

[0117] The processing system 300 is configured to modify each training input image using the value of each device characteristic to generate a modified training data set. The processing system can also use the modified training data set to train a machine learning algorithm.

[0118] In a first embodiment, the processing system 300 includes a cloud computing device 310 and a mobile / cellular device 320. The mobile / cellular device is configured to receive, for example, the value(s) of the device characteristic(s) from a personal care device 350. The mobile / cellular device then passes the received value(s) onto the cloud computing system to perform the remaining steps of the method.

[0119] In a second embodiment, the processing system 300 includes only the cloud computing device 310. The cloud computing device can be configured to directly receive, for example, the value(s) of the device characteristic(s) from the personal care device 350 before performing the remaining steps of the method.

[0120] Alternative methods for receiving / obtaining at least one of the values (instead of from the personal care device) are also contemplated. For example, at least one of the values can be stored by another memory 340 (e.g., a cloud computing-based memory system). It is possible to use the device identifier of the personal care device (e.g., the model and / or brand of the personal care device) to retrieve the value(s).

[0121] It should be understood that a computer-implemented method for performing a desired task on an input image generated by a camera of a personal care device is also proposed.

[0122] The computer-implemented method includes: obtaining an input image; and processing the input image using a machine learning algorithm that is trained using the method described previously (i.e., trained using the modified training data set generated by the described embodiments) to perform a desired task on the input image.

[0123] Reference Figure 3 , the computer-implemented method can be executed by the processing system 300. In particular, the computer-implemented method can be executed by the mobile device 320 and / or the cloud computing system 310. Thus, the mobile device 320 and / or the cloud computing system can host the trained machine learning algorithm (regardless of where the algorithm is trained).

[0124] In one example, a machine learning algorithm is trained by a cloud computing system 310 and deployed to a mobile / cellular device 320. The mobile / cellular device can then acquire an input image and use the machine learning algorithm to process the input image to perform a desired task.

[0125] In another example, a machine learning algorithm is trained by a cloud computing device and deployed in a cloud computing system 310. The input image can be passed to the cloud computing system (optionally via the mobile / cellular device 320) and processed by the cloud computing system to perform a desired task.

[0126] The result of the desired task is some output data, such as classification, etc. If generated by the cloud computing system, the output data can be provided to the mobile / cellular device 320 (e.g., for display) or another electronic device, such as a terminal 330 for healthcare professionals (e.g., a laptop / computer) or a storage / memory system, such as a separate memory 340.

[0127] A training data set generated or obtainable by performing any of the previously described methods is also provided, which is used to modify a training data set available for training a machine learning algorithm to perform a desired task on an input image generated by a camera of a personal care device. Similarly, a machine learning algorithm trained using such a training data set is also provided.

[0128] The training data set and / or the machine learning algorithm can be stored on a storage medium, and preferably on a non-transitory storage medium. Accordingly, a non-transitory storage medium storing the training data set and / or the machine learning algorithm is provided.

[0129] Those skilled in the art will be able to easily develop a processing system for performing any of the methods described herein. Accordingly, each step of the flowchart can represent a different action performed by the processing system and can be performed by a corresponding module of the processing system.

[0130] The processing system can be implemented in various ways using software and / or hardware to perform the various functions required. The processing system typically employs one or more microprocessing systems, which can be programmed using software (e.g., microcode) to perform the required functions. The processing system can be implemented as a combination of dedicated hardware for performing some functions and one or more programmed microprocessing systems and associated circuitry for performing other functions.

[0131] Examples of circuitry that can be employed in various embodiments of the present disclosure include, but are not limited to, conventional microprocessing systems, application specific integrated circuits (ASICs), and field programmable gate arrays (FPGAs).

[0132] In various embodiments, a processing system may be associated with one or more storage media, such as volatile and non-volatile computer memories, such as RAM, PROM, EPROM, and EEPROM. The storage media may be encoded with one or more programs that, when executed on one or more processing systems and / or controllers, perform the required functions. The various storage media may be fixed within the processing system or controller, or may be transportable, such that one or more programs stored thereon may be loaded into the processing system.

[0133] A computer program product is also proposed, which includes computer program code portions that, when executed by a processing system, cause the processing system to perform all of the steps of any of the methods disclosed herein. The computer program may be stored / distributed on any suitable non-transitory medium, such as an optical storage medium or a solid-state medium provided together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems.

[0134] Based on the study of the drawings, the disclosure, and the appended claims, those skilled in the art can understand and implement variations of the disclosed embodiments when practicing the claimed invention. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" does not exclude a plurality.

[0135] The functions implemented by the processing system may be implemented by a single processing system or multiple separate processing units, which may together be considered to constitute a "processing system". Such processing units may in some cases be remote from each other and communicate with each other in a wired or wireless manner.

[0136] The fact that certain measures are recited only in mutually different dependent claims does not indicate that a combination of these measures cannot be used advantageously.

[0137] If the term "adapted to" is used in the claims or the specification, otherwise it should be noted that the term "adapted to" is intended to be equivalent to the term "configured to". If the term "arranged" is used in the claims or the specification, otherwise it should be noted that the term "arranged" is intended to be equivalent to the term "system", and vice versa.

[0138] Any reference signs in the claims shall not be construed as limiting the scope.

Claims

1. A computer-implemented method for modifying a training data set that can be used to train a machine learning algorithm to perform a desired task on input images generated by a camera of a personal care device, the computer-implemented method comprises: for each device characteristic among one or more device characteristics of the personal care device, obtaining a value of the device characteristic; obtaining the training data set for the machine learning algorithm, the training data set including a plurality of training data entries, each training data entry including a training input image and training output data, the training output data representing data generated by performing the desired task on the training input image; and characterized in that: using the value of each device characteristic to modify each training input image to generate a modified training data set.

2. The computer-implemented method according to claim 1, wherein the value of at least one device characteristic is generated by one or more sensors of the personal care device.

3. The computer-implemented method according to any one of claims 1 to 2, wherein at least one of the one or more device characteristics responds to or indicates movement of the personal care device during use.

4. The computer-implemented method according to claim 3, wherein the one or more device characteristics include one or more of the following: the vibration frequency of the personal care device and / or a vibration element of the personal care device during use; the vibration amplitude of the personal care device and / or a vibration element of the personal care device during use; the vibration waveform or pattern of the personal care device and / or a vibration element of the personal care device during use; the rotation frequency of a rotation element of the personal care device during use; and / or the rotation waveform or pattern of a rotation element of the personal care device during use.

5. The computer-implemented method according to any one of claims 1 to 4, wherein at least one of the one or more device characteristics is a characteristic of the camera of the personal care device.

6. The computer-implemented method according to claim 5, wherein the one or more device characteristics include one or more of the following: the frame rate of the camera or any other imaging sensor; an indicator of whether the camera uses global shutter technology; an indicator of whether the camera uses rolling shutter technology; if the camera uses rolling shutter technology, the line collection time and / or the time delay between exposures of adjacent image lines; if the camera uses global shutter technology, the duty cycle of image acquisition; and / or one or more lighting parameters of the camera.

7. The computer-implemented method according to any one of claims 1 to 6, further comprises: the step of obtaining the value of each user characteristic among one or more user characteristics of the user of the personal care device, wherein the step of modifying each training input image includes: using the values of each device characteristic and each user characteristic to modify each training input image to generate the modified training data set.

8. The computer-implemented method according to any one of claims 1 to 7, further comprises: using the modified training dataset to train the machine learning algorithm.

9. A computer-implemented method for performing a desired task on an input image generated by a camera of a personal care device, the computer-implemented method comprises: obtaining the input image; and using a machine learning algorithm to process the input image, the machine learning algorithm being trained using the method according to claim 8 to perform the desired task on the input image.

10. A training dataset that can be obtained by performing the method according to any one of claims 1 to 7.

11. The computer-implemented method according to any one of claims 1 to 10, wherein the method is executed by a processing system separated from the personal care device, such as a cloud computing processing system.

12. A computer program product comprising computer program code portions that, when executed by a processing system, cause the processing system to perform all steps of the method according to any one of claims 1 to 11.

13. A processing system for modifying a training dataset available for training a machine learning algorithm to perform a desired task on an input image generated by a camera of a personal care device, the processing system comprises: for each device characteristic among one or more device characteristics of the personal care device, obtaining the value of the device characteristic; obtaining the training dataset for the machine learning algorithm, the training dataset including a plurality of training data entries, each training data entry including a training input image and training output data, the training output data representing data generated by performing the desired task on the training input image; and characterized in that: using the value of each device characteristic to modify each training input image to generate a modified training dataset.

14. The processing system according to claim 13, wherein the value of each device characteristic is generated by one or more sensors of the personal care device.

15. The processing system according to claim 13 or 14, wherein at least one of the one or more device characteristics is: movement in response to or indicating the personal care device during use; and / or characteristics of the camera of the personal care device.

Images