Positioning method for personal care device

Abstract

A method of locating at least a portion of a personal care device in real time relative to an anatomy of a user based on using an artificial intelligence prediction model. The predictive model is configured to receive as input a starting location of at least a portion of the personal care device.

Description

Technical Field

[0001] The present invention relates to a method for real-time positioning of a part of a personal care device during use. Background Art

[0002] In prior art personal care devices, there is a need for means to track the position of the operating part of the personal care device relative to the user's body. For example, in the case of an oral care device, there is a need to track the position of the brush head within the user's mouth.

[0003] There are currently several smart toothbrushes that incorporate such functionality. For example, smart electric toothbrushes can typically provide feedback to the user regarding brushing behavior, such as feedback on daily brushing frequency and mouth segment coverage. Additionally, electric toothbrushes typically include a variety of sensors integrated in the brush head and / or handle. Based on the sensor data, it is possible to infer various attributes, including the positioning of the brush head inside the mouth.

[0004] To provide the user with insights related to previous brushing sessions, an app (application) running on a mobile computing device can be used, which can display a mouth map indicating to the user whether sufficient time has been spent cleaning each mouth segment. The application can also be configured to aggregate data over a longer time span (e.g., weeks, months, etc.). Some connected electric toothbrush applications can also provide insights regarding mouth positions where too much pressure and / or scrubbing has been applied.

[0005] Similar smart features may also be used for other personal care devices, such as shavers, skin care devices, massage devices, light therapy devices, etc.

[0006] One method for performing real-time position tracking of a personal care device is to use data from an inertial measurement unit (IMU), which can include, for example, accelerometer data and gyroscope data. Algorithms capable of mapping the IMU data to a position are used.

[0007] However, it is a challenge to provide a position estimate based on linear mapping of IMU data. In particular, there is sometimes no one-to-one mapping between device orientation and position. For example, in the case of an oral care device, some mouth segments can cause ambiguity or confusion. For instance, brushing the right outer molars and the left inner molars results in similar handle orientations.

[0008] It has previously been recognized that it is difficult to determine the currently being brushed segment solely from the IMU signal without considering the history of previously brushed segments since the start of the personal care session.

[0009] Thus, in more recent developments in this field, algorithms have emerged that can consider the recent history of the positioning location since the start of a personal care session. This helps narrow down the scope of the real-time location. However, as a result of doing so, the positioning accuracy of the real-time prediction model is generally lower at the start of a brushing session than later in the brushing session because the history of the previous location is greatly reduced. Summary of the Invention

[0010] The present invention is defined by the claims.

[0011] According to an example of an aspect of the present invention, a method for real-time positioning of at least a portion of a personal care device during a personal care session is provided. The method includes: obtaining data indicating a starting position of the portion of the personal care device for the personal care session; receiving real-time inertial measurement unit (IMU) signal data during the personal care session; retrieving a real-time positioning model from a data store, wherein the positioning model includes an artificial intelligence (AI) model configured to receive a first input and a second input and generate a real-time positioning signal indicating a real-time predicted position of the device portion as an output, the first input being data indicating the starting position of the at least a portion of the personal care device, and the second input including the real-time IMU signal data; providing the obtained starting position of the device portion as an input to the positioning model; during the personal care session, providing the received real-time IMU signal data for the personal care session as an input to the positioning model; receiving the output real-time positioning signal from the positioning model; and preferably generating a data output based on the real-time positioning signal.

[0012] Thus, embodiments of the present invention propose to improve the positioning accuracy, particularly at the start of a personal care session, by providing a positioning prediction model configured to receive the starting position of at least a portion of the personal care device as an input. The model can be configured to use the starting position as an additional initialization variable for the positioning model, thereby allowing the positioning model to be initialized differently depending on the starting position. This allows the AI model to be configured differently depending on the starting position, allowing for improved positioning accuracy, particularly at the start of the session.

[0013] In some embodiments, the AI model included in the positioning model is an artificial neural network. In a preferred embodiment, the AI model included in the positioning model is a recurrent artificial neural network.

[0014] In some embodiments, the AI model is a long short-term memory (LSTM) artificial neural network. The LSTM neural network is a type of recurrent neural network that includes a cell state and a hidden state, where the cell state acts as a more persistent (long) "memory" of the network. Both the hidden state and the cell state are passed from each recurrent processing step to the next. This will be explained in more detail later.

[0015] In some embodiments, the personal care device is an oral care device, and the method is for real-time positioning of the brush head portion of the oral care device within the user's mouth during a cleaning session. However, the method can also be used for any other type of personal care device, such as a hair removal device (e.g., a razor), a skin care device, a light therapy device, a massage device, etc.

[0016] In some embodiments, the step of obtaining data indicative of a starting position includes receiving, from a user interface, user input indicative of a planned starting position. In other words, the user specifies the planned starting position.

[0017] In some embodiments, the user interface can be a user interface included in a mobile computing device.

[0018] In some embodiments, the step of obtaining data indicative of a starting position includes accessing a data store that records historical personal care data for the user and determining a predicted starting position based on processing of the historical personal care data.

[0019] In some embodiments, the historical personal care data for the user includes historical starting position data, and the predicted starting position includes processing of the historical starting position data.

[0020] For example, in some embodiments, the aforementioned historical personal care data includes IMU signal data of historical personal care sessions, and predicting the starting position of a personal care session includes: processing the historical IMU signal data to estimate the historical starting position and predicting the starting position based on the estimated historical starting position. In some embodiments, the method can further include generating a control signal for controlling the user interface to generate a user-perceivable prompt requesting confirmation of the predicted starting position.

[0021] In some embodiments, the method further includes: receiving a user setting for the starting position; comparing the user setting for the starting position with the predicted starting position; and in response to detecting a difference between the two, generating a control signal for controlling the user interface to generate a user-perceivable prompt requesting confirmation of the predicted starting position.

[0022] In some embodiments, the method may be performed in a first mode or a second mode, where in the first mode, positioning is performed using knowledge of the starting position, and in the second mode, positioning is performed without using knowledge of the starting position. For example, the method may further include a preliminary mode selection step that includes selecting the first mode or the second mode; wherein in response to the selection of the first mode, the steps of the method according to any of the above example embodiments are performed, and in response to the selection of the second mode, a second positioning module is retrieved, the second positioning module being configured to perform real-time positioning without an input indicating the starting position and providing real-time IMU signal data as an input to the second positioning model.

[0023] In some embodiments, a single positioning model may be provided that can operate in two different states: one designed to receive an input indicating the starting position and one designed to operate without receiving such an input. For example, according to one or more embodiments, the positioning model may be initialized in two states: a first state in which the positioning model is configured to generate positioning data based on using an input indicating the starting position; and a second state in which the positioning model is configured to generate positioning data without using an input indicating the starting position. The method may further include a preliminary mode selection step that includes selecting the first mode or the second mode. In response to the selection of the first mode, the positioning model may be used in the first initialization state, and in response to the selection of the second mode, the positioning model may be used in the second initialization state.

[0024] In some embodiments, the positioning model includes an LSTM artificial neural network, i.e., the AI model of the positioning model includes an LSTM artificial neural network. The method may include: depending on a first input including data indicating the starting position of the portion of the personal care device, initializing the cell state and the hidden state of the LSTM artificial neural network.

[0025] In some embodiments, the positioning model includes at least one embedding layer configured to map a first input indicating the starting position to at least one embedding vector, and the at least one embedding layer is configured to initialize the cell state and the hidden state of the LSTM artificial neural network based on the at least one embedding vector.

[0026] In some embodiments, the positioning model includes at least two embedding layers, one embedding layer for outputting an embedding vector for initializing the hidden state based on an input indicating the starting position, and one embedding layer for outputting an embedding vector for initializing the cell state based on an input indicating the starting position.

[0027] In some embodiments, the starting position may be encoded in the form of an integer having a limited range of possible values. In some embodiments, different values of the integer may correspond to different body segment positions, such as different mouth segment positions.

[0028] In some embodiments, the IMU signal data may include 3D accelerometer signal data and 3D gyroscope sensor data.

[0029] Another aspect of the present invention is a computer program product comprising computer program code configured to, when run on a processor, cause the processor to perform a method according to any of the embodiments described herein or according to any of the claims of the present application. The code may be configured to cause the processor to perform the method when the processor is operably coupled to an inertial measurement unit (IMU) included in a personal care device and operably coupled to a data store storing data including a real-time positioning model including an artificial intelligence (AI) model, the positioning model being configured to receive a first input and a second input and generate a real-time positioning signal indicative of a real-time predicted position of a device portion as an output, the first input being data indicative of a starting position of at least a portion of the personal care device, the second input including real-time IMU signal data. Another aspect of the present invention is a processing device for real-time positioning of at least a portion of a personal care device during a personal care session, the processing device comprising: an input / output; and one or more processors configured to perform a method. The method may include: obtaining data indicative of a starting position of the portion of the personal care device for a personal care session; receiving, during the personal care session, real-time inertial measurement unit (IMU) signal data at the input / output; retrieving a real-time positioning model from a data store, wherein the positioning model includes an artificial intelligence (AI) model, the positioning model being trained to receive a first input and a second input and generate a real-time positioning signal indicative of a real-time predicted position of a device portion as an output, the first input including data indicative of a starting position of the portion of the personal care device, the second input including real-time IMU signal data; providing the obtained starting position of the head portion as an input to the positioning model; during the personal care session, providing the received real-time IMU signal data for the personal care session as an input to the positioning model; receiving an output real-time positioning signal from the positioning model; and preferably generating a data output based on the real-time positioning signal and optionally coupling the data output to the input / output.

[0030] Another aspect of the present invention provides a system that includes: a personal care device that includes an integrated IMU for generating IMU signal data during a personal care session and includes a wireless communication module for transmitting the IMU signal; and a processing device according to any embodiment described herein or according to any claim of the present application, the processing device being configured to receive the IMU signal data transmitted by the personal care device. In some embodiments, the personal care device is an oral care device, such as an electric toothbrush.

[0031] The IMU signal data can be received directly from the personal care device via a direct wireless communication channel, such as in the case where the processing device consists of a mobile computing device. The mobile computing device and the personal care device can be in a communicable relationship via a local wireless connection (e.g., Bluetooth or another technology). Alternatively, the IMU sensor data can be received via an intermediate communication channel, such as via an Internet connection or a local area network connection. In some embodiments, the processing device can be a cloud-based processing device.

[0032] In some embodiments, the system includes a mobile communication device that includes a user interface.

[0033] In some embodiments, the processing device can be included as a component of the mobile communication device.

[0034] Alternatively, the processing device can be external to and separated from the mobile communication device, such as the processing device can be in the cloud.

[0035] These and other aspects of the present invention will become apparent and be elucidated with reference to the embodiments (one or more) described below. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] To better understand the present invention and to more clearly show how to implement the present invention, reference will now be made, by way of example only, to the accompanying drawings, in which:

[0037] Figure 1 The architecture of an example basic neural network model for positioning a portion of a personal care device is shown;

[0038] Figure 2 Examples of encodings of different segments of a mouth as part of an example implementation of a positioning method are illustrated;

[0039] Figure 3 The accuracy of a positioning prediction generated by the basic neural network model is shown, which is a function of time elapsed since the start of a personal care session;

[0040] Figure 4Outlines the steps of an example method according to one or more embodiments of the present invention;

[0041] Figure 5 Outlines the components of an example processing device and system according to one or more embodiments of the present invention;

[0042] Figure 6 Outlines the processing flow according to one or more embodiments of the present invention;

[0043] Figure 7 Illustrates the architecture of an example neural network model according to one or more embodiments of the present invention; and

[0044] Figure 8 Illustrates the accuracy of a positioning prediction generated by a neural network model according to one or more embodiments of the present invention. Detailed Description

[0045] The present invention will be described with reference to the accompanying drawings.

[0046] It should be understood that the detailed description and specific examples, although indicating exemplary embodiments of the apparatus, system and method, are for illustrative purposes only and are not intended to limit the scope of the present invention. These and other features, aspects and advantages of the apparatus, system and method of the present invention will become better understood from the following description, the appended claims and the drawings. It should be understood that the drawings are merely schematic and are not drawn to scale. It should also be understood that in all the drawings, the same reference numerals are used to indicate the same or similar parts.

[0047] The present invention provides an improved method for real-time positioning of at least a part of a personal care device relative to a user's anatomy using an artificial intelligence prediction model. To improve the accuracy of positioning, especially at the beginning of a personal care session, the prediction model is configured to receive the starting position of at least a part of the personal care device as an input. The prediction model can be initialized differently based on the received starting position. This compensates for the lack of information about the previously positioned location in the session encountered at the beginning of the session.

[0048] For ease of explanation and brevity, the following explanation may refer specifically to an oral care device, and more particularly, to tracking the position of a brush head within a user's mouth. However, it should be understood that the general principles of positioning a part of a device relative to an anatomical region can be applied, without loss of generality, to various different personal care devices.

[0049] As described above, estimating the position of a personal care device relative to an anatomical structure based on IMU data (e.g., estimating the position of a brush head inside the mouth) is a challenging problem. For example, some mouth segments can cause ambiguity or confusion. For instance, brushing the right outer molar and the left inner molar produces similar handle orientations. Without considering the session history of previously brushed segments since the start of the session, it is difficult to determine the currently brushed segment from the IMU signal.

[0050] In other words, in order to robustly detect the current brushing position in real time, the model requires context information (such as brushing movements and / or mouth segment transitions). However, such information is lacking at the beginning of a brushing session.

[0051] To further understand the principles of the currently proposed inventive concept, examples of basic positioning algorithms that do not conform to the present invention will now be described. Some specific embodiments of the present invention represent the development of this algorithm. In particular, the features described with respect to this model can also be applied to the model to be referred to later Figure 7 described.

[0052] The model architecture is as Figure 1 shown. The model includes an AI-enabled algorithm (AI model) that is operable to process sensor data and generate real-time predictions of the position of relevant parts of the personal care device relative to the user's anatomical structure. The AI model includes an artificial neural network, and in particular a recurrent neural network.

[0053] Figure 1 The block diagram of shows the basic network architecture of such a real-time brush head positioning model. The model in this example includes a convolutional long short-term memory (LSTM) artificial neural network architecture.

[0054] The LSTM artificial neural network is a type of recurrent artificial neural network. Like other types of recurrent neural networks, it is capable of processing not only discrete data items such as images, but also extended data sequences such as data signals. This makes it very suitable for processing real-time IMU data signals in the current environment. Compared with other types of recurrent neural networks, its network structure encompasses both feedback connections and feedforward connections. A standard recurrent neural network can be understood as containing short-term memory and long-term memory, where the connection weights and biases in the network represent long-term memory and are updated only once for each training set, and the activation patterns in the network represent short-term memory and change at each time interval within the input data. The basic concept of a long short-term memory network is to implement short-term memory that lasts for more than one time step in the network.

[0055] The neural network model may include one or more LSTM units. Generally, an LSTM unit has a structure including an LSTM unit, where the unit includes an input gate, an output gate, and a forget gate. The gates have the function of controlling the inflow and outflow of data into and out of the unit. The LSTM unit receives the latest versions of the hidden state and the cell state as inputs, and generates updated versions of the hidden state and the cell state as outputs. The cell state provides a storage function, where data values are "remembered" over an arbitrary period of time. This allows the network to process extended data sequences, where related events can occur at arbitrary time intervals. Therefore, having a short-term type of memory that can persist over an arbitrary period of time allows such patterns to be detected.

[0056] As is well known, during a prediction operation, a recurrent neural network operates by recursively processing the hidden state, which contains a set of variables, and the hidden state is passed as an output from one step in the network processing to the next step. The LSTM additionally uses the cell state, which carries "memory" information that persists longer than the hidden state. At each processing step of the network, the LSTM takes the hidden state and the cell state output from the previous step as inputs, and is used to generate updated hidden and cell states.

[0057] Therefore, the hidden state refers to a set of variables used in the recurrent operation of the neural network, and is actually the input to the work done in the model at a given step, and is calculated based on the data from the previous time step. It can be understood as the result of a hidden layer and is then passed as an input to the next hidden layer.

[0058] The cell state effectively acts as a memory, and carries information all the way along the processing chain, and is updated using the various gates (forget gate, input gate, and output gate) of the LSTM unit at each processing step. The cell state helps the network consider long-term dependencies. The cell state can be analogized to a conveyor belt; it runs straight through the entire network with only some minor linear interactions. In contrast, the hidden state can be analogized to a set of buckets. The hidden state at time t is a set of values that represent what the network "remembers" at that time. The hidden state is reset at the start of each sequence, so it does not contain any information about the previous sequence.

[0059] Regarding training, a model covering one or more LSTM network units can be trained, for example, using a supervised learning method. For example, a training data set including a set of training sequences can be used. Each training sequence can include an example signal data sequence. For example, in this context, the training data can include a set of example IMU data signals. An optimization algorithm such as the gradient descent algorithm can be employed in combination with backpropagation over time to calculate the gradients required during the optimization process. The aim is to change each weight of the LSTM network in proportion to the derivative of the error (at the output layer) with respect to the relevant weights.

[0060] For additional details regarding LSTM neural networks, see the publicly accessible literature: Hochreiter, Sepp, and Jürgen Schmidhuber, "Long Short-Term Memory," Neural Computation 9.8 (1997): 1735-1780. This provides a detailed explanation of the structure and implementation of a suitable LSTM network. In particular, Sections 2 and 3 of this article provide the background of the special challenges that LSTM networks can overcome compared to other types of networks. Section 4 discusses the general structure of LSTM networks. Section 5 describes a series of example implementations and associated results.

[0061] Referring again to Figure 1 the example neural network model of, the input data 102 of the model includes real-time IMU data signals. The first component of the model includes an encoder 104 having a single 1D convolutional block or layer 122 (followed by a batch normalization ("Batchnorm") layer 124). During training, the encoder 102 is able to learn a feature representation of the input 102 sensor data. The feature mapping representation is captured in the feature map. After training, the encoder encodes the input sensor data 102 based on the feature representation learned during training.

[0062] The output of the encoder 104 is fed into a single-layer long short-term memory (LSTM) neural network unit 106. During training, the LSTM unit is able to model the temporal dependencies between the feature map activations.

[0063] The output of the LSTM block (i.e., the final hidden state) is input to the classifier 108 head, which includes a 1D convolutional layer 154 preceded by two 1D convolutional blocks 150, 152. As additional background, in a neural network, the term "hidden layer" is used to refer to a layer located between the input and output of an algorithm, where a function applies weights to the input and guides them through an activation function as output. In contrast, in the context of a recurrent neural network, "hidden state" means a set of variables used in the recurrent operation of the network. The hidden state is actually the input to what is done in the model at a given step and is calculated based on the data from the previous time step. This hidden state can be understood as the result of a hidden layer and is then passed as input to the next hidden layer.

[0064] Each of the convolutional blocks 150, 152 includes a respective convolutional layer 132, 138, followed by batch normalization layers 134, 140, followed by rectified linear unit (ReLu) layers 136, 142. The final convolutional layer group 154 includes a convolutional layer 144 combined with a batch normalization layer 146. It should also be noted that other configurations are possible, for example, having a different number of layers in the encoder 104, LSTM 106, and classification module 108.

[0065] The output 110 of the model generated by the classifier module 108 using the LSTM 106 output includes a respective probability prediction for each position segment in a plurality of position (sub) segments of the relevant anatomical structure at each time point of the real-time input data signal 102. From these sub-segments, the most likely one position segment of the current position of the relevant device part can be identified.

[0066] By way of example, an exemplary embodiment of the model is generated and trained to predict the real-time position of the brush head within the mouth. To this end, the classifier module 108 is configured to generate 12 (probability) prediction outputs, one for each of the 12 position sub-segments of the mouth. Figure 2 Schematically illustrates the 12 position sub-segments of the mouth. The numbered layers correspond as follows:

[0067] 0: Upper right outer

[0068] 1: Front upper outer

[0069] 2: Upper left outer

[0070] 3: Lower left outer

[0071] 4: Front lower outer

[0072] 5: Lower right outer

[0073] 6: Upper right inner

[0074] 7: Front upper inner

[0075] 8: Upper left inner

[0076] 9: Lower left inner

[0077] 10: Front lower inner

[0078] 11: Lower right inner

[0079] Using such a prediction model, the positioning accuracy is generally lower at the start of a brushing session compared to later time periods in the session. By this time, the user has been brushing for some time, and thus there is a larger amount of recent history from which to generate predictions. Due to the memory function of the LSTM units discussed above, once some time has passed in a personal care session, the model is able to account for the temporal dependencies in the input data.

[0080] To verify this hypothesis, a baseline model was constructed, trained, and tested using a holdout / test dataset consisting of the IMU data signals from 1056 previously recorded brushing sessions. Figure 1 of the baseline model.

[0081] The results are shown in Figure 3 a box plot of.

[0082] It can be observed that there is a large variation in the positioning accuracy at the start of the brushing session. Each box plot indicates the positioning accuracy (y-axis) within the first n seconds from the start of the brushing segment (x-axis). The last box plot represents the average accuracy over a full cleaning session.

[0083] By way of example, it can be observed that the median accuracy within the first n = 2 seconds is approximately 78%. Additionally, the box Figure Four whiskers indicate a large range around the mean, especially in the lower quartiles (i.e., the first and second quartiles). The accuracy box plot for a longer time period from the start of the session (e.g., n = 90 seconds) indicates less variation in the lower quartiles and a slightly higher median accuracy of approximately 82%. This indicates that the prediction accuracy improves over time since the start of the session. After a longer time period, the model has access to a larger amount of context regarding the movement history of the brushing session to estimate the current segment being brushed.

[0084] Therefore, the inventors recognized that the explanation for the low performance at the start of the session is that Figure 1 the real-time positioning model does not utilize any prior knowledge about the actual starting position of the brush head inside the mouth (i.e., the mouth segment in the case of Figure 1 the model). The same principle will apply to any personal care device being tracked relative to any relevant anatomical structure.

[0085] Embodiments of the present invention propose to solve the above problems by using an AI model configured to infer the real-time current position of at least a portion of a personal care device, and wherein the inference is conditioned on information about the starting position of the at least a portion of the personal care device.

[0086] According to a set of specific embodiments of an example of the present invention, it is proposed by modifying Figure 1The basic model architecture of the model is used to solve the above problems, such that the LSTM module of the model is adapted to receive a representation of the starting position of the tracked portion of the personal care device as an additional input. This can be provided as an input to the localization model at the start of inference (i.e., at the start of the personal care session). By way of an example, in the case of tracking the position of the brush head in the mouth, the starting position of the brush head (at the start of the session) can be encoded as one of the mouth segments for which the classifier module generates a prediction output (e.g., encoded as an integer value between 0 - 11).

[0087] A set of specific embodiments of the present invention propose using a recurrent artificial neural network including at least one LSTM unit to infer the real - time current position of at least a part of a personal care device, wherein the inference is conditioned on the starting position of the at least a part of the personal care device. However, more generally, any AI model trained to receive the starting position as an input can be used.

[0088] Figure 4 The steps of an example method 10 according to one or more embodiments are outlined in block diagram form. These steps will be outlined before being further explained in the form of example embodiments.

[0089] Method 10 is for real - time localizing a part of a personal care device during a personal care session.

[0090] Method 10 includes obtaining or receiving 12 data indicating the starting position of the part of the personal care device for the personal care session.

[0091] Method 10 further includes receiving 14 real - time inertial measurement unit (IMU) signal data during the personal care session.

[0092] Method 10 further includes retrieving 16 a real - time localization model from a data store. The localization model includes an artificial intelligence (AI) model, which is trained to receive a first input and a second input, the first input including data indicating the starting position of the part of the personal care device, the second input including real - time IMU signal data, and generating a real - time localization signal indicating the real - time predicted position of the device part as an output.

[0093] Method 10 further includes providing 18 the obtained starting position of the device part as an input to the localization model.

[0094] Method 10 further includes providing 20 the received real - time IMU signal data of the personal care session as an input to the localization model during the personal care session.

[0095] Method 10 further includes receiving 22 the output real - time localization signal from the localization model.

[0096] Method 10 preferably further includes generating a data output 24 based on real-time positioning signals.

[0097] As described above, the method may also be embodied in hardware form, for example, in the form of a processing device configured to execute the method according to any example or embodiment described in this document or according to any claim of this application.

[0098] For further assistance in understanding, Figure 5 A schematic representation of an example processing device 32 configured to execute the method according to one or more embodiments of the present invention is presented. The processing device is shown in the context of a system 30 that includes the processing device. The processing device represents one aspect of the present invention alone. The system 30 is another aspect of the present invention. The provided system need not include all of the shown hardware components; it may be just a subset thereof.

[0099] The processing device 32 includes one or more processors 36 configured to execute the method according to the method outlined above, or according to any embodiment described in this document or any claim of this application. In the example shown, the processing device further includes an input / output 34 or a communication interface.

[0100] In Figure 5 the example shown, the system 30 further includes a user interface 52. In some embodiments, the positioning signal may be transmitted to the user interface for transmitting positioning information to the user. In some embodiments, as will be described below, an indication of the starting position may be input by the user at the user interface 52 and received by the processing device 32 and fed as input to the positioning model. In some embodiments, the user interface may be included in a mobile computing device such as a smartphone. In some embodiments, the user interface may be included in a personal care device.

[0101] The system 30 in this example further includes the personal care device 54 itself. The personal care device 54 includes an inertial measurement unit (IMU) 56. The IMU may include one or more accelerometers for generating accelerometer data. The IMU may include at least one gyroscope for generating gyroscope data. In some embodiments, the IMU signal data output from the IMU may include accelerometer data and / or gyroscope data. In some embodiments, the IMU signal data output from the IMU may include 3D acceleration measurements and 3D gyroscope sensor readings. The IMU signal data 44 may be transmitted from the IMU of the personal care device 54 to the processing device 32 for providing as input to the positioning model.

[0102] The system 30 in this example further includes a data store 58 that stores the localization model 60. The method executed by the processing device 32 may include retrieving the localization model 60 from the data store.

[0103] The present invention may also be embodied in software form. Accordingly, another aspect of the present invention is a computer program product comprising computer program code configured to, when run on a processor, cause the processor to execute a method according to any of the embodiments described herein or according to any of the claims of the present application.

[0104] It should be noted that, referring again to Figure 5 the system 30, the system or processing unit 32 may include a memory 38 that stores computer program instructions executed by one or more processors 36 of the processing unit to cause the one or more processors to execute a method according to any one of the embodiments described in the present disclosure.

[0105] Depending on the architecture of the system, the processing of the sensor data 44 from the IMU 56 can be performed in several ways. For example, in some embodiments, the method of the present invention may be implemented by a software module. The software module may be implemented by a processing device included in a mobile computing device, or by a personal care device, or by a separate processing device, or may be implemented by a remote server or a cloud server. By way of only one example, the IMU sensor data may be synchronized with a smartphone app, and a software module integrated in the app may run method 10 to infer the brush head position using the sensor data. Alternatively, the software for brush head positioning may be embedded in the personal care device itself, such as in the handle of the device. Alternatively, the sensor data may be synchronized with a cloud infrastructure, and the software module for executing method 10 is also executed as part of the cloud infrastructure. The output from the method may be transmitted to the user and presented using a user interface, such as an app run by a smartphone or a user interface included in the personal care device.

[0106] One application area is the oral care field. In some embodiments, the personal care device 54 is an oral care device, and the personal care session is an oral cleaning session. At least a portion of the tracked device may include, for example, a brush head. In other embodiments, the personal care device may be a skin care device, such as a phototherapy device, and the personal care action is a skin care action, and the personal care session is a skin care session. At least a portion of the device may be an effective skin treatment portion, such as a light output area in the case of a phototherapy device. In other embodiments, the personal care device is a combing device, such as a razor, and the personal care action is a combing action (e.g., shaving), and the personal care session is a combing session (e.g., shaving). At least a portion of the tracked device may include, for example, a razor head. In other embodiments, the personal care device is a muscle / joint care device (e.g., a massage device). In each case, the personal care device is a device for applying to an area of ​​the user's body to achieve a care function, whereby the normal use of the device inherently involves moving the device around different locations of one or more areas / anatomical structures of the body.

[0107] To illustrate and illustrate the concept of the present invention, an example about an oral care device can be given below, for which the aforementioned personal care action is a cleaning action, and the personal care session is an oral cleaning session. However, it will be appreciated that the principles outlined can be easily applied to other types of personal care devices, such as, by skin care devices, such as phototherapy devices, beauty devices, such as shavers, or muscle / joint care devices (e.g., massage devices). For any personal care device, the device is generally designed to be applied to an area of ​​the user's body to achieve a care function, and can be moved around in different positions. Therefore, it can be seen how the principles of the overall inventive concept outlined above and described below can be used for any personal care device. Therefore, in the following embodiments, references to oral care devices can be replaced by references to any other personal care device without causing substantial changes to the inventive principle. Similarly, references to cleaning actions can be replaced by references to personal care actions, and references to cleaning sessions can be replaced by references to personal care sessions.

[0108] Where the personal care device 54 is an oral care device, in some embodiments, the oral care device may be an electric toothbrush. A toothbrush may have a brush head that carries a series of cleaning elements, such as bristles.

[0109] The data processing flow according to one or more embodiments is Figure 6is shown in schematic form. As shown, the localization model 60 is configured to receive at least two data inputs. The first data input includes data representing a starting position 64 of a personal care device portion for a personal care session. The second input includes real-time inertial measurement unit (IMU) signal data 44 for the personal care session. The localization model 60 generates a real-time localization signal 68 as an output.

[0110] Regarding the localization model, as described above, this includes an artificial intelligence model. The AI model can be an artificial neural network. In a preferred embodiment, the AI model is a recurrent artificial neural network. The recurrent neural network can be an LSTM artificial neural network.

[0111] Now will refer to Figure 7 describe an example implementation of the localization model according to one or more embodiments. Except that the LSTM unit 106 is configured to receive an additional input indicating the starting position 64 of at least a portion of the personal care device being localized and utilize the input starting position in the localization inference, the model structure can be the same as Figure 1 the structure of the model (described above). In some embodiments, this can be achieved through learnable embeddings 162, as will now be explained.

[0112] Specifically, in some embodiments, the localization model includes an LSTM artificial neural network, and wherein the method includes: initializing the cell state and the hidden state of the LSTM artificial neural network depending on a first input including data indicating the starting position of the portion of the personal care device. More specifically, in some embodiments, the localization model includes at least one embedding layer configured to map the input starting position to at least one embedding vector and configured to initialize the cell state and the hidden state based on the at least one embedding vector. Even more specifically, in some embodiments, the localization model includes at least two embedding layers, one embedding layer for outputting an embedding vector for initializing the hidden state based on an input indicating the starting position, and one embedding layer for outputting an embedding vector for initializing the cell state based on an input indicating the starting position.

[0113] Now this will be explained in more detail.

[0114] In the context of neural networks, an embedding is a low-dimensional vector representation of the relationships present in high-dimensional input data. More specifically, an embedding is a mapping of discrete (categorical) variables to continuous numeric vectors. In the context of neural networks, an embedding is a low-dimensional, learned continuous vector representation of discrete variables. The distance between embedding vectors captures the similarity between different data points and can capture higher-level underlying concepts in the original input. Neural network embeddings are actually an inherent byproduct of the normal supervised training process. In particular, an embedding is a vector representation of the network parameters (weights) that are adjusted during training to minimize the loss of the supervised task. The resulting embedding vectors are representations of classes, where similar classes - relative to the task - are closer to each other. In this document, the task is the localization of at least a portion of a personal care device. The classes can be different starting positions, for example encoded with one of a set of discrete integers corresponding to different position segments. Thus, in the context of the present invention, each possible starting position will have an embedding vector (e.g., in the case of a discrete number of starting positions, encoded with an integer corresponding to a particular part of an anatomical structure). In operation, the embedding layer will provide the function of mapping an input indicator of the starting position (e.g., in the form of an integer) to the corresponding one of a set of learned embedding vectors, and where the corresponding learned embedding vector can then be used as the initialization of the hidden state and / or cell state of the LSTM. As described below, there can be two embedding layers, one for outputting the learned embedding vector that initializes the cell state and one for outputting the learned embedding vector that initializes the hidden state.

[0115] As described above, a recurrent neural network operates by recursively processing the hidden state, which contains a set of variables, and the hidden state is passed as output from one step in the network processing to the next. The LSTM additionally uses a cell state, which carries "memory" information that persists longer than the hidden state. Each processing step of the LSTM takes as input the hidden state and cell state output from the previous step and is used to generate updated hidden and cell states.

[0116] Thus, the hidden state refers to a set of variables used in the recurrent operation of a neural network and is actually the input to the work done in the model at a given step and is computed based on data from previous time steps. It can be understood as the result of one hidden layer and is then passed as input to the next hidden layer.

[0117] The cell state effectively acts as a memory and carries information all the way along the processing chain and is updated at each processing step using the various gates (forget gate, input gate, and output gate) of the LSTM cell.

[0118] In the context of some embodiments of the present invention, each embedding vector thus has the same size / dimension as the hidden state or the cell state, respectively.

[0119] Given an input start position, e.g., encoded by an integer, the model learns the latent embedding vectors in an end-to-end manner during training while training the remaining model parameters. In particular, at the start of training, the relevant start position of the input given training data signal (e.g., in the form of an integer) is given, and this is passed as input to two different embedding layers: one embedding layer for the hidden state and one embedding layer for the cell state. To explain: Generally speaking, inside an LSTM there are two states, namely the hidden state and the cell state. Therefore, it is proposed to use two embedding layers to learn and condition each of the mentioned states based on the start position.

[0120] Each embedding layer maps the start position (e.g., an integer) to a latent vector of the same size as the LSTM hidden state or cell state. During training and inference, the LSTM model utilizes such latent embeddings to predict real-time localization in the form of a class label associated with one of a discrete set of possible positions.

[0121] During operation, for inference, the learned embeddings effectively act as lookup tables, i.e., map the input start position to the corresponding embedding vector for each of the hidden state and the cell state. The embedding vectors are then used to define the initialization of the hidden state and the cell state. More specifically, this means that the starting state of the hidden state is set to the value defined by the hidden state embedding vector output from the respective embedding layer, and the starting state of the cell state is set to the value defined by the cell state embedding vector output from the respective embedding layer.

[0122] Thus, the embeddings are learned during model training (and the embedding layers are part of the model), and can map an input start position (e.g., represented as an integer) to a real-valued vector output that is used as the initialization of the hidden and cell states of the LSTM model, which would otherwise be initialized to zero at the start of the inference procedure. As part of the network operation, during model operation, the hidden and cell states are updated iteratively.

[0123] It should be noted that if no start position is provided, there are two options for how to initialize the hidden and cell states: a default initialization can be used, which corresponds, for example, to an “average” start position based on global statistics (e.g., the most frequent start position in the general population), or the LSTM hidden and cell states can be initialized with zeros.

[0124] Figure 7 The high-level model architecture is shown in.

[0125] In addition to the additional use of the learned embedding 162 corresponding to different possible input start positions 64 of at least a part of the personal care device, the structure of the model can be the same as that of Figure 1 (details are discussed above). The embedding corresponding to the input start position 64 is used to initialize the hidden and cell states of the LSTM cell 106 of the neural network at the start of the personal care session.

[0126] Thus, embodiments of the present invention propose integrating the start position to develop a conditional variant of the real-time segmentation mode, aiming to improve the model generalization ability at the start of the session. For this purpose, it is proposed to have a learnable embedding vector 162 for each of the possible start positions (i.e., output classes), which can be used as the initialization of the hidden and cell states of the LSTM.

[0127] This minimal but effective method significantly improves the model generalization ability without overhead.

[0128] It should be noted that any of the implementation details of the model description regarding Figure 1 can also be applied to the model of Figure 7 , except that the model of Figure 7 additionally includes one or more embedding layers. The model of Figure 1 is similar to the model of

[0129] The above only represents one possible way to provide the model, where the start position can be used as an input. Alternative ways of integrating the start position in the deep neural network model are possible.

[0130] According to an alternative, in an LSTM-based model, instead of having separate learnable embedding layers for the hidden and cell states, a single embedding layer can be shared to further reduce the model size. The dimensions of these states are the same; thus, the output of the single embedding layer can be used as the initialization of both states. However, as mentioned above, the hidden and cell states are meant to learn different things and control different aspects of the LSTM cell, so using two embedding layers may be preferred.

[0131] In addition, additionally or alternatively, instead of using the latent embedding vector as the initialization of the LSTM state, it is also an option to adjust the output of the LSTM cell 106 by summing it with the latent vector. In this case, the embedding layer provides an output with the same dimension as the LSTM output. For example, at the start of training, when defining the neural network architecture, the LSTM output can be set to size 128, and the embedding layer output size can also be set to 128. Now, we can sum the outputs of the LSTM layer and the embedding layer, simply performing an element-wise sum of the vectors. In a deep learning library such as Pytorch, this can be achieved via the torch.add function.

[0132] In addition, additionally or alternatively, Feature-wise Linear Modulation (FiLM) can be incorporated at the level of the convolutional encoder 104 to adjust the features in the early layers of the model. For feature-wise linear modulation, reference can be made to the paper "FiLM: Visual Reasoning with a General Conditioning Layer" by Perez, Ethan, et al., Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32, No. 1, 2018.

[0133] Formally, Feature-wise Linear Modulation is a general conditioning layer that modulates the input features. The conditioning layer is a neural network that takes as input the features of another layer (e.g., the encoder) and the output of the previous FiLM layer (if available), and outputs a set of modulation parameters. These modulation parameters are then used to modulate the input features (by performing element-wise multiplication and addition) to produce a new feature map. FiLM layers can be stacked on top of other layers to form a deep FiLM network.

[0134] The FiLM layer is useful because it allows the network to learn how to modulate the features of the input signal to produce the desired output. For example, if the goal is to incorporate the starting position to make the network more likely to correctly predict the real-time positioning location at the start of a personal care session, the FiLM layer can learn to modulate the features of the previous layer such that the starting position is incorporated into the features to increase the likelihood of the desired output.

[0135] At the implementation level, the output of the convolutional encoder 104 (as Figure 1 or Figure 7 shown) is passed as input, together with the output of the embedding layer used as a conditioning mechanism, to the FiLM layer. The FiLM layer performs modulation of the encoder features based on the embedding vector representing the starting position. Finally, its output is further passed to the LSTM.

[0136] It should be noted that the FiLM layer acts like any other layer in a deep model and includes a single linear layer to transform the conditional input (in this case, the output of the embedding layer), which is used to update the encoder output via element-wise multiplication and addition operations.

[0137] We note that the above methods can also be combined together to further enhance the model performance.

[0138] As described above, in advantageous embodiments, the starting position is encoded as an integer having a defined range of possible values.

[0139] The different values of the integer can correspond to different body segment positions, such as different mouth segment positions.

[0140] Figure 8 Illustrated is the model performance of the proposed invention, which uses the same test dataset for testing the performance of the basic model outlined in Figure 1 and the results are shown in Figure 3 To test the model, the model is provided as input for each cleaning session: the real-time IMU signal data for that session, and the true starting position for that session derived from reference data. Comparing the results of the basic model in Figure 3 with the results of the model proposed in Figure 8 it is clear that for the proposed model, both the average positioning accuracy and the range of positioning accuracy about the average are significantly improved. In particular, for time periods close to the start of a personal care session (e.g., 2 seconds, 5 seconds, 10 seconds from the start), the average accuracy and the accuracy range are greatly improved. Thus, this shows that additionally using the starting position as a conditional initialization parameter in the model improves the positioning accuracy towards the start of a personal care session.

[0141] Some optional implementation features of method 10 in operation will now be discussed.

[0142] To implement the method, a personal care device such as an electric toothbrush can be provided with electronics to allow synchronization with a software module (such as an app) running on a computing device such as a mobile computing device or running in the cloud. The personal care device includes an IMU sensor as described above, for example configured with a three-axis accelerometer and a three-axis gyroscope.

[0143] In some embodiments, the app may prompt the user to input a preferred or planned starting position of the personal care device, such as a planned segment in the mouth where they plan to start brushing. In other words, in some embodiments, obtaining data indicating the starting position includes receiving user input indicating the planned starting position from a user interface. The starting position may be added to the settings dataset. Optionally, the settings dataset may additionally be transmitted to the personal care device, for example via a wireless connection such as Bluetooth.

[0144] A user interface may be provided that allows the user to input, such as the planned starting position.

[0145] Optionally, in some embodiments, the software module may include one or more algorithms configured to automatically determine a predicted starting position based on the user's personal care device usage history, such as a brushing history. In other words, obtaining data indicating the starting position may include accessing a data store recording historical personal care data for the user and determining the predicted starting position based on processing of the historical personal care data. If the starting position is determined in this way, the user interface may also be controlled to provide a prompt that allows the user to confirm or adjust the automatically determined starting position. In other words, the method may include generating a control signal for controlling the user interface to generate a user-perceivable prompt requesting confirmation of the predicted starting position.

[0146] A common way to implement the user interface (UI) is to use an App, such as an App run by a mobile computing device such as a smartphone. In other words, the user interface may be the user interface included by the mobile computing device. However, this is by no means the only way to implement the UI. The UI may also be implemented in the cloud and even in the personal care device, depending on the processing power and connectivity.

[0147] Regarding the execution of the pre-trained localization model, this may be implemented, for example, by a processing module included in a mobile computing device, or by a processing module of a cloud computing architecture, or by a processing module of the personal care device. The model outputs a real-time localization signal indicating the real-time position of the personal care device, such as the position of the brush head in the mouth.

[0148] At the start of a brushing session, the user can be prompted by a cue generated by the user interface to position at least a portion of the personal care device at an indicated starting position. The system can be configured such that the start of the session is triggered by a user activation signal generated by actuation of a user control element (such as the user pressing a power button on the personal care device). At the start of the session, IMU signal data can be sampled from the sensor unit and provided in real time as input, along with the starting position of the device, to a localization model. The model is initialized with the starting position information and then starts real-time localization inference.

[0149] Optionally, additional software modules can be included as part of the system that receive the current predicted position of at least a portion of the personal care device as input and calculate a real-time coverage estimate of a relevant anatomical structure or region of the anatomical structure. For example, in the case of an oral care device, such a software module can be configured to update the current coverage estimate of a particular mouth segment that is currently being cleaned (e.g., the elapsed brushing session).

[0150] In some embodiments, the personal care device can include one or more indicator elements, such as indicator lights, for conveying the real-time coverage of the current anatomical region, e.g., the brushing coverage of the mouth segment currently being brushed. For instance, the indicator light can include a color-coded light emitted by a light ring integrated on the surface of the personal care device housing.

[0151] In some embodiments, instead of requiring the user to provide a preferred starting position, alternatively, a dedicated software module can be utilized to predict / retrieve the most likely starting position by analyzing starting position estimates derived from previous sessions performed by the user.

[0152] In other words, and as a more general principle, in some embodiments, the step of obtaining data indicative of the starting position can include accessing a data store that records historical personal care data for the user and determining a predicted starting position based on processing of the historical personal care data.

[0153] In this case, by way of an example, a post-brush localization model (also referred to as an offline model) can be used to derive the predicted starting position, which model runs, for example, in a cloud environment and can access the raw IMU data of previous sessions stored in the same cloud infrastructure. Compared to the real-time prediction configuration, the offline prediction model can generally have higher accuracy because the post-brush prediction model is not constrained by any causal correlations but is able to consider both backward and forward temporal correlations. Except that the LSTM layer is modified to be bidirectional such that it can perform both forward and backward inference, the offline prediction model can be provided as having the same as Figure 7structures and operations that are substantially the same as the model. This allows the model to infer backwards to determine the prediction of the starting position. Alternatively, the offline prediction model can be set up with an Figure 7 architecture different from that of Figure 7 . An example is a UNet-based localization model. The offline model can be configured to process the IMU data of multiple personal care sessions and export the predicted "most frequent" starting position. This type of method can produce a more accurate prediction of the most likely starting position because it can process the entire IMU dataset of one or more personal care sessions offline after the relevant session ends.

[0154] In other words, in some embodiments, the historical personal care data may include the IMU signal data of historical personal care sessions, and predicting the starting position of a personal care session includes: processing the historical IMU signal data to estimate the historical starting position, and predicting the starting position based on the estimated historical starting position.

[0155] As a simpler implementation, the historical cleaning personal care data for a user may include historical starting position data, and predicting the starting position includes processing the historical starting position data.

[0156] In some embodiments, the method may confirm the predicted starting position with the user. In other words, in some embodiments, the method further includes generating a control signal for controlling the user interface to generate a user-perceivable prompt requesting confirmation of the predicted starting position.

[0157] In some embodiments, once the most likely start segment of the next brushing session is retrieved from the historical session data for the user, the software module can compare it with the start segment of the current setting recorded in the settings dataset. If there is a difference, the software app can generate a message dialog at the user interface asking the user if they want to update the starting position of the current setting.

[0158] In other words, as a more general principle, in some embodiments, the method includes additionally receiving the user's setting of the starting position; comparing the user's setting for the starting position with the predicted starting position; and in response to detecting a difference between the two, generating a control signal for controlling the user interface to generate a user-perceivable prompt requesting confirmation of the predicted starting position.

[0159] According to at least one set of embodiments, the provision of the starting position can be optional, which means that the system can include two positioning models dedicated to two specific tasks. When no starting position is provided or the starting position is not yet known, the first model can be trained end-to-end, specifically for positioning. When the user has provided a planned starting position or a reliable user-specific starting position has been inferred from the historical session data for the user, the second model can be trained end-to-end specifically for positioning. Depending on whether a preferred starting point has been provided or made available, the solution selects an appropriate model for localization inference.

[0160] In other words, in some embodiments, the method may further include a preliminary mode selection step that includes selecting a first mode or a second mode. One of the modes is configured for positioning using the input starting position, and the other mode is configured for positioning without using the input starting position. For example, in response to the selection of the first mode, the steps of the method according to the previously described embodiments are applied. For example, this can be a model according to Figure 7 . In response to the selection of the second mode, a second positioning model can be retrieved that is configured to perform real-time positioning without an input indicating the starting position, and the real-time IMU signal data is provided as input to the second positioning model. For example, this can be a model according to Figure 1 .

[0161] Another option is to provide a system where the provision of the starting position is optional, but where the system includes a single positioning model capable of handling two cases: the first case where the user has not provided a preferred starting position or it is not yet known, and the second case where a preferred starting position is provided or estimated based on the historical session data for the user.

[0162] In other words, in some embodiments, a single positioning model can be initialized in two states: a first state where the model is configured to generate positioning data based on using an input indicating the starting position, and a second state where the model is configured to generate positioning data without using an input indicating the starting position. The method may further include a preliminary mode selection step that includes selecting a first mode or a second mode, and where in response to the selection of the first mode, the positioning model is used in the first initialization state, and in response to the selection of the second mode, the positioning model is used in the second initialization state.

[0163] In this method, the model is initialized differently depending on whether a starting position is provided. For example, in the case where the starting position is known, the model is initialized by selecting a specific embedding value associated with the provided starting position, which determines the LSTM hidden and cell states and is obtained from training. Otherwise, when the starting position is unknown, the model is initialized by assigning all-zero hidden and cell state vectors for the LSTM. It should be noted that such models that can handle both cases can be trained end-to-end.

[0164] The embodiments of the present invention described above employ a processing device. Generally speaking, a processing device can include a single processor or multiple processors. It can be located in a single containing device, structure, or unit, or it can be distributed among multiple different devices, structures, or units. Thus, a reference to a processing device suitable for or configured to perform a particular step or task can correspond to a step or task performed by any one or more of a plurality of processing components, either individually or in combination. Those skilled in the art will understand how to implement such a distributed processing device. The processing device can include a communication module or input / output for receiving data and outputting data to other components.

[0165] One or more processors of the processing device can be implemented in software and / or hardware in various ways to perform the various functions required. Processors typically employ one or more microprocessors, which can be programmed using software (such as microcode) to perform the required functions. A processor can be implemented as a combination of dedicated hardware for performing some functions and one or more programmed microprocessors and associated circuitry for performing other functions.

[0166] Examples of circuitry that can be used in various embodiments of the present disclosure include, but are not limited to, conventional microprocessors, application-specific integrated circuits (ASICs), and field-programmable gate arrays (FPGAs).

[0167] In various embodiments, a processor can be associated with one or more storage media, such as volatile and non-volatile computer memories, such as RAM, PROM, EPROM, and EEPROM. The storage media can be encoded with one or more programs that, when executed on one or more processors and / or controllers, perform the required functions. The various storage media can be fixed within the processor or controller, or they can be removable, such that one or more programs stored thereon can be loaded into the processor.

[0168] Based on a study of the drawings, the disclosure, and the appended claims, those skilled in the art can understand and implement variations of the disclosed embodiments when practicing the claimed invention. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality.

[0169] A single processor or other unit may implement the functions of several items recited in the claims.

[0170] The fact that certain measures are recited only in mutually different dependent claims does not indicate that a combination of these measures cannot be used advantageously.

[0171] A computer program may be stored / distributed on a suitable medium, such as an optical storage medium or a solid-state medium supplied with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems.

[0172] If the term "adapted to" is used in a claim or specification, it should be noted that the term "adapted to" is intended to be equivalent to the term "configured to".

[0173] Any reference signs in the claims shall not be construed as limiting the scope.

Claims

1. A method (10) for real-time positioning of at least a part of a personal care device (54) during a personal care session, comprising: obtaining (12) data indicating a starting position of the at least a part of the personal care device for the personal care session; receiving (14) real-time inertial measurement unit (56) IMU signal data during the personal care session; retrieving (16) a real-time positioning model (60) from a data store (58), wherein the positioning model includes an artificial intelligence AI model, the positioning model being trained to receive a first input and a second input and generate a real-time positioning signal indicating a real-time predicted position of the device part as an output, the first input including data indicating the starting position of the at least a part of the personal care device, and the second input including the real-time IMU signal data; providing (18) the obtained starting position of the device part as an input to the positioning model; during the course of the personal care session, providing (20) the received real-time IMU signal data for the personal care session as an input to the positioning model; receiving (22) an output real-time positioning signal from the positioning model; and generating (24) a data output based on the real-time positioning signal.

2. The method according to claim 1, wherein the AI model is an artificial neural network, and preferably a recurrent neural network.

3. The method according to claim 1 or 2, wherein the AI model is a long short-term memory LSTM artificial neural network.

4. The method according to any one of claims 1 to 3, wherein the personal care device is an oral care device, and the method is for real-time positioning of a head part of the oral care device within a user's mouth during a cleaning session.

5. The method according to any one of claims 1 to 4, wherein obtaining data indicating the starting position includes receiving user input indicating a planned starting position from a user interface, and optionally, wherein the user interface is a user interface included in a mobile computing device.

6. The method according to any one of claims 1 to 4, wherein obtaining data indicating the starting position comprises: accessing a data store recording historical personal care data for a user and determining a predicted starting position based on processing of the historical personal care data.

7. The method according to claim 6, wherein the historical personal care data for the user includes historical starting position data, and predicting the starting position includes processing of the historical starting position data, and / or wherein the historical personal care data includes IMU signal data for historical personal care sessions, and wherein predicting the starting position of the personal care session comprises: processing the historical IMU signal data to estimate a historical starting position and predicting the starting position based on the estimated historical starting position.

8. The method according to claim 7, wherein the method further comprises: receiving a user setting for the starting position; comparing the user setting of the starting position with the predicted starting position; and In response to detecting a difference therebetween, a control signal for controlling a user interface is generated to generate a user-perceivable prompt requesting confirmation of the predicted starting position.

9. The method according to any one of claims 1 to 8, wherein the method further comprises a preliminary mode selection step, the preliminary mode selection step comprising selecting a first mode or a second mode; wherein in response to the selection of the first mode, the steps of the method according to any one of claims 1 to 8 are performed, and in response to the selection of the second mode, a second positioning module is retrieved, the second positioning module being configured to perform real-time positioning without an input indicating a starting position and providing the real-time IMU signal data as an input to the second positioning model.

10. The method according to any one of claims 1 to 8, wherein the positioning model can be initialized in two states: a first state, wherein the positioning model is configured to generate positioning data based on using an input indicating a starting position; and a second state, wherein the positioning model is configured to generate positioning data without using an input indicating a starting position; wherein the method further comprises a preliminary mode selection step, the preliminary mode selection step comprising selecting a first mode or a second mode; wherein in response to the selection of the first mode, the positioning model is used in the first initialization state, and in response to the selection of the second mode, the positioning model is used in the second initialization state.

11. The method according to any one of claims 1 to 10, wherein the starting position is encoded in the form of an integer having a defined range of possible values, and optionally, wherein different values of the integer correspond to different body segment positions, such as different mouth segment positions.

12. The method according to any one of claims 1 to 11, wherein the AI model comprises a long short-term memory (LSTM) artificial neural network, and wherein the method comprises: initializing a cell state and a hidden state of the LSTM artificial neural network depending on the first input, the first input comprising the data indicating the starting position of the part of the personal care device.

13. The method according to claim 12, wherein the positioning model comprises at least one embedding layer, the at least one embedding layer being configured to map the first input indicating the starting position to at least one embedding vector, and the at least one embedding layer being configured to initialize the cell state and the hidden state of the LSTM artificial neural network based on the at least one embedding vector.

14. The method according to claim 13, wherein the positioning model comprises at least two embedding layers, one embedding layer for outputting an embedding vector for initializing the hidden state based on the input indicating the starting position, and one embedding layer for outputting an embedding vector for initializing the cell state based on the input indicating the starting position.

15. A computer program product comprising computer program code configured to, when run on a processor, cause the processor to perform the method according to any one of claims 1 to 14.

16. A processing device (32) for real-time positioning of at least a part of a personal care device (54) during a personal care session, comprising: input / output (34); and one or more processors (36), the one or more processors being configured to execute a method comprising: obtaining data indicative of a starting position of the at least a part of the personal care device for a personal care session; during the personal care session, receiving real-time inertial measurement unit IMU signal data at the input / output; retrieving from a data store (58) a real-time positioning model (60), wherein the positioning model comprises an artificial intelligence AI model, the positioning model being trained to receive a first input and a second input and generate as output a real-time positioning signal indicative of a real-time predicted position of the device part, the first input comprising data indicative of the starting position of the part of the personal care device, the second input comprising real-time IMU signal data; providing the obtained starting position of the head part as an input to the positioning model; during the course of the personal care session, providing the received real-time IMU signal data for the personal care session as an input to the positioning model; and receiving from the positioning model an output real-time positioning signal; and generating a data output based on the real-time positioning signal and optionally coupling the data output to the input / output.

17. A system (30), comprising: a personal care device (54), the personal care device comprising an integrated IMU for generating IMU signal data during a personal care session and comprising a wireless communication module for transmitting the IMU signals; and the processing device (32) according to claim 16, the processing device being configured to receive the IMU signal data transmitted by the personal care device, and optionally, wherein the system comprises a mobile communication device, the mobile communication device comprising a user interface.

Images