Method for localizing personal care devices

Abstract

A method for real-time localization of at least a portion of a personal care device relative to a user's anatomy is based on the use of an artificial intelligence predictive model, where the predictive model is configured to receive as input a starting position of at least a portion of the personal care device.

Description

[Technical Field]

[0001] The present invention relates to a method for real-time localization of parts of a personal care device in use. [Background technology]

[0002] State-of-the-art personal care devices require a means of tracking the position of the working parts of the personal care device relative to the user's anatomy, for example, in the context of oral care devices, it is desirable to track the position of the brush head within the user's mouth.

[0003] Currently, several smart toothbrushes are available that feature this functionality. For example, smart electric toothbrushes typically provide users with feedback on their brushing behavior, such as daily brushing frequency and coverage of oral cavity segments. Furthermore, electric toothbrushes typically include various sensors integrated into the brush head and / or handle. From the sensor data, various attributes can be inferred, including the position of the brush head within the oral cavity.

[0004] To provide users with insight into previous brushing sessions, apps executed by the mobile computing device can be used that display a mouse map that indicates to the user whether enough time was spent cleaning each oral cavity segment. The app can also be configured to aggregate data over longer time spans (e.g., weeks, months, etc.). Some connected power toothbrush applications can also provide insight into areas of the mouth where too much pressure and / or excessive scrubbing was applied.

[0005] Similar smart features are possible in other personal care devices, such as shavers, skin care devices, massage devices, and light therapy devices. Summary of the Invention [Problem to be solved by the invention]

[0006] One approach to performing real-time location tracking of a personal care device is to use data from an inertial measurement unit (IMU), including, for example, accelerometer and gyroscope data. Algorithms are used that can map the IMU data to a location.

[0007] However, providing position estimation based on linear mapping from IMU data is challenging. In particular, there may not be a one-to-one correspondence between device orientation and position. For example, in the context of oral care devices, some oral segments may be ambiguous or confusing. For example, brushing the right outer molar and the left inner molar produces similar brushing handle orientations.

[0008] It has previously been recognized that it is difficult to determine the currently brushed segment from the IMU signal alone without considering the history of previously brushed segments since the start of the personal care session.

[0009] Therefore, new developments in this area seek to create algorithms that can take into account the recent history of localization positions since the beginning of a personal care session, which helps refine the real-time position. However, this results in the localization accuracy of real-time predictive models being generally lower at the beginning of a brushing session compared to later periods in the brushing session, because there is significantly less previous position history. [Means for solving the problem]

[0010] The invention is defined by the claims.

[0011] According to an example according to one aspect of the present invention, there is provided a method for real-time localization of at least a portion of a personal care device during a personal care session, the method comprising the steps of: obtaining data indicative of a start position of a portion of the personal care device for the personal care session; receiving real-time inertial measurement unit (IMU) signal data during the personal care session; obtaining a real-time localization model from a data store, the localization model including an artificial intelligence (AI) model, the localization model configured to receive a first input including data indicative of a start position of at least the portion of the personal care device and a second input including real-time IMU signal data, and to generate as an output a real-time localization signal indicative of a real-time predicted position of the portion of the device; providing the obtained start position of the portion of the device as an input to the localization model; providing the received real-time IMU signal data related to the personal care session as input to the localization model over the course of the personal care session; receiving the output real-time localization signal from the localization model; and preferably generating a data output based on the real-time localization signal.

[0012] Therefore, embodiments of the present invention propose to improve localization accuracy, particularly at the beginning of a personal care session, by providing a localization prediction model configured to receive as input a starting position of at least a portion of a personal care device. The model is configured to use the starting position as an additional initialization variable for the localization model, which allows the localization model to be initialized differently depending on the starting position. This allows the AI ​​model to be configured differently depending on the starting position, which allows for improved localization accuracy, particularly at the beginning of a session.

[0013] In some embodiments, the AI ​​model included in the localization model is an artificial neural network. In a preferred embodiment, the AI ​​model included in the localization model is a recurrent artificial neural network.

[0014] In some embodiments, the AI ​​model is a long-short-term memory (LSTM) artificial neural network. LSTM neural networks are a class of recurrent neural networks that contain a cell state and a hidden state, with the cell state serving as the network's more persistent (longer) "memory." Both the hidden state and the cell state are passed from each recurrent processing step to the next. This is explained in more detail below.

[0015] In some embodiments, the personal care device is an oral care device and the method is for real-time localization of a brush head portion of the oral care device within a user's oral cavity during a cleaning session, however, the method can also be used with other types of personal care devices, such as hair removal devices (such as shavers), skin care devices, phototherapy devices, massage devices, etc.

[0016] In some embodiments, obtaining data indicative of the start position comprises receiving user input from a user interface indicative of the intended start position, in other words, a user specifying the intended start position.

[0017] In some embodiments, the user interface may be a user interface included in a mobile computing device.

[0018] In some embodiments, obtaining data indicative of the start location includes accessing a data store that records historical personal care data of the user and determining a predicted start location based on processing the historical personal care data.

[0019] In some embodiments, the user's historical personal care data includes historical start location data, and predicting the start location comprises processing the historical start location data.

[0020] For example, in some embodiments, the historical personal care data includes IMU signal data of past personal care sessions, and predicting a start location for the personal care sessions includes processing the historical IMU signal data to estimate past start locations and predicting the start location based on the estimated past start locations. In some embodiments, the method may further include generating a control signal to control a user interface to generate a user-perceptible prompt requesting confirmation of the predicted start location.

[0021] In some embodiments, the method further includes receiving a user setting of a start location; comparing the user setting of the start location with the predicted start location; and, based on detecting a difference between the two, generating a control signal for controlling a user interface to generate a user-perceptible prompt requesting confirmation of the predicted start location.

[0022] In some embodiments, the method can be performed in either a first mode or a second mode, where in the first mode localization is performed using knowledge of a starting location and in the second mode localization is performed without knowledge of a starting location. For example, the method can further include a preliminary mode selection step including selecting the first or second mode, where based on the selection of the first mode, the steps of the method according to any of the exemplary embodiments described above are performed, and based on the selection of the second mode, a second localization module configured to perform real-time localization without input indicating a starting location is obtained; and providing real-time IMU signal data as input to the second localization model.

[0023] In some embodiments, a single localization model may be provided that is operable in two different states: one designed to receive an input indicating a starting location and one designed to operate without receiving such input. For example, according to one or more embodiments, the localization model may be initialized in two states: a first state in which the localization model is configured to generate localization data based on using an input indicating a starting location, and a second state in which the localization model is configured to generate localization data without using an input indicating a starting location. The method may further include a preliminary mode selection step including selecting the first mode or the second mode. Based on the selection of the first mode, the localization model may be used in the first initialization state, and based on the selection of the second mode, the localization model may be used in the second initialization state.

[0024] In some embodiments, the localization model comprises an LSTM artificial neural network, i.e., the AI ​​model of the localization model comprises an LSTM artificial neural network. The method may comprise initializing cell states and hidden states of the LSTM artificial neural network based on a first input comprising data indicative of a starting position of the portion of the personal care device.

[0025] In some embodiments, the localization model has at least one embedding layer configured to map a first input indicating a starting location to at least one embedding vector, and is configured to initialize cell states and hidden states of the LSTM artificial neural network based on the at least one embedding vector.

[0026] In some embodiments, the localization model has at least two embedding layers: one embedding layer for outputting embedding vectors for initializing the hidden state based on inputs indicating a starting location, and one embedding layer for outputting embedding vectors for initializing the cell state based on inputs indicating a starting location.

[0027] In some embodiments, the start position may be encoded in the form of an integer value with a predetermined range of possible values, in which different values ​​of the integer value may correspond to different body segment positions, for example different oral cavity segment positions.

[0028] In some embodiments, the IMU signal data may include 3D accelerometer signal data and 3D gyroscope sensor data.

[0029] Another aspect of the present invention is a computer program having computer program code configured, when executed on a processor, to cause the processor to perform a method according to any embodiment described herein or according to any claim of the present application. The code may be configured to cause the processor to perform the method when the processor is operatively coupled to an inertial measurement unit (IMU) included in a personal care device and a data store storing a real-time localization model including an artificial intelligence (AI) model, the localization model configured to receive a first input including data indicative of a starting position of at least a portion of the personal care device and a second input including real-time IMU signal data, and to generate as an output a real-time localization signal indicative of a real-time predicted position of the device portion. Another aspect of the present invention is a processing device for use in real-time localization of at least a portion of a personal care device during a personal care session, the processing device having an input / output and one or more processors configured to perform a method.The method includes the steps of obtaining data indicating a starting position of a portion of the personal care device for a personal care session, receiving real-time inertial measurement unit (IMU) signal data at an input / output during the personal care session, obtaining a real-time localization model from a data store, the localization model including an artificial intelligence (AI) model, the localization model receiving a first input including data indicating a starting position of the portion of the personal care device and a second input including real-time IMU signal data, and being trained to generate as an output a real-time localization signal indicating a real-time predicted position of the portion of the device, providing the obtained starting position of the head portion as an input to the localization model, providing the received real-time IMU signal data related to the personal care session as an input to the localization model over the course of the personal care session, receiving the output real-time localization signal from the localization model, and preferably generating a data output based on the real-time localization signal and optionally coupling the data output to the input / output.

[0030] Another aspect of the present invention provides a system, comprising: a personal care device including an integrated IMU for generating IMU signal data during a personal care session, the personal care device including a wireless communication module for transmitting the IMU signals; and a processing device according to any embodiment described herein or any claim herein, configured to receive the IMU signal data transmitted by the personal care device. In some embodiments, the personal care device is an oral care device, such as an electric toothbrush.

[0031] The IMU signal data can be received directly from the personal care device via a direct wireless communication channel, for example, when the processing device is included in a mobile computing device. The mobile computing device and the personal care device are in communicative relationship via a local wireless connection, such as Bluetooth or another technology. Alternatively, the IMU sensor data can be received via an intermediate communication channel, such as via an internet connection or a local area network connection. In some embodiments, the processing device can be a cloud-based processing device.

[0032] In some embodiments, the system comprises a mobile communication device that includes a user interface.

[0033] The processing device may, in some embodiments, be included as part of a mobile communications device.

[0034] Alternatively, the processing unit may be external to and separate from the mobile communications device, for example, the processing unit may be in the cloud.

[0035] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter. [Brief explanation of the drawings]

[0036] [Figure 1] FIG. 1 illustrates an exemplary basic neural network model architecture for localizing parts of a personal care device. [Figure 2] FIG. 1 illustrates an exemplary encoding of different segments of the mouth as part of an exemplary implementation of a localization method. [Figure 3] FIG. 10 shows the accuracy of localization predictions produced by a basic neural network model as a function of time elapsed since the start of the personal care session. [Figure 4]FIG. 1 outlines steps of an exemplary method in accordance with one or more embodiments of the present invention. [Figure 5] FIG. 1 is a diagram outlining elements of an exemplary processing device and system in accordance with one or more embodiments of the present invention. [Figure 6] FIG. 1 is a diagram outlining a process flow according to one or more embodiments of the present invention. [Figure 7] FIG. 1 illustrates the architecture of an exemplary neural network model in accordance with one or more embodiments of the present invention. [Figure 8] FIG. 10 illustrates the accuracy of localization predictions produced by a neural network model in accordance with one or more embodiments of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0037] For a better understanding of the present invention, and in order to show more clearly how it may be carried into effect, reference will now be made to the accompanying drawings, which are given by way of example only, in which:

[0038] The present invention will now be described with reference to the figures.

[0039] It should be understood that the detailed description and specific examples, while indicating exemplary embodiments of the devices, systems, and methods, are for purposes of illustration only and are not intended to limit the scope of the invention. These and other features, aspects, and advantages of the devices, systems, and methods of the present invention will be better understood from the following description, appended claims, and accompanying drawings. It should be understood that the figures are schematic only and are not drawn to scale. It should also be understood that the same reference numerals are used throughout the figures to indicate the same or similar parts.

[0040] The present invention provides an improved method for real-time localization of at least a portion of a personal care device relative to a user's anatomy based on the use of an artificial intelligence predictive model. To improve the accuracy of the localization, especially at the beginning of a personal care session, the predictive model is configured to receive as input a starting position of at least a portion of the personal care device. The predictive model can be initialized differently based on the received starting position. This compensates for the lack of information about previous localization positions within the session encountered at the beginning of the session.

[0041] For ease and brevity, the following description may be presented with particular reference to tracking the position of an oral care device, and more particularly, a brush head within a user's oral cavity, although it should be understood that the general principles of localizing portions of a device relative to an anatomical region can be applied to a wide variety of different personal care devices without loss of generality.

[0042] As mentioned above, estimating the position of a personal care device relative to anatomical structures (e.g., estimating the position of a brush head within the oral cavity) based on IMU data is a challenging problem. For example, some oral cavity segments may be ambiguous or confusing. For example, brushing the right outer molar and the left inner molar produces similar brushing handle directions. It is difficult to determine the currently brushed segment from the IMU signal without considering the session history of previously brushed segments since the start of the session.

[0043] In other words, to robustly detect the current brushing position in real time, the model requires contextual information (e.g., brushing motion and / or mouth segment transitions), which is lacking at the beginning of a brushing session.

[0044] To further understand the principles underlying the presently proposed inventive concept, an example of a basic localization algorithm not in accordance with the present invention will now be described. Some specific embodiments of the present invention represent developments of this algorithm. In particular, the features described in relation to this model can also be applied to the model described later with reference to FIG. 7.

[0045] The model architecture is outlined in Figure 1. The model comprises an AI-enabled algorithm (AI model) operable to process sensor data and generate real-time predictions of the position of relevant parts of the personal care device relative to the user's anatomy. The AI ​​model comprises an artificial neural network, and in particular a recurrent neural network.

[0046] The block diagram in Figure 1 shows the base network architecture of this real-time brushhead localization model. The model in this example has a convolutional long-short-term memory (LSTM) artificial neural network architecture.

[0047] The LSTM artificial neural network is a type of recurrent artificial neural network. Like other types of recurrent neural networks, it can process not only discrete data items like images but also extended sequences of data, such as data signals. This makes the model suitable for processing real-time IMU data signals, in this context. In contrast to other types of recurrent neural networks, its network structure includes not only feedforward connections but also feedback connections. A standard recurrent neural network can be understood as encompassing both short-term and long-term memory. The connection weights and biases within the network represent long-term memory and are updated only once per training episode, while the activation patterns within the network represent short-term memory and are changed once per time interval in the input data. The concept underlying long-short-term memory networks is to achieve short-term memory within the network that lasts longer than one time step.

[0048] A neural network model has one or more LSTM units. Typically, an LSTM unit has a structure containing an LSTM cell, which has an input gate, an output gate, and a forget gate. The gates control the flow of data into and out of the cell. An LSTM cell receives the latest version of the hidden state and cell state as input and produces updated versions of the hidden state and cell state as output. The cell state provides memory functionality, and data values ​​can be "remembered" for any period of time. This allows the network to handle extended data sequences, where related events can occur at any time interval. Therefore, having a short-term memory that can persist for any period of time allows such patterns to be detected.

[0049] As is well known, during prediction operations, recurrent neural networks operate by recursively processing a hidden state, which includes a set of variables and is passed as output from one step in the network's processing to the next. LSTMs also use cell states, which hold "memory" information that persists for a longer time than the hidden state. At each processing step of the network, the LSTM acts as an input, taking the hidden state and cell state output from the previous step and generating updated hidden and cell states.

[0050] Thus, a hidden state refers to the set of variables used in the recurrent operation of a neural network; it is effectively the input to what is happening in the model at a given time step, calculated based on the data from previous time steps. It can be understood as the result of one hidden layer being passed as input to the next hidden layer.

[0051] The cell state effectively acts as a memory, carrying information throughout the processing chain and is updated at each processing step using the various gates of the LSTM cell (forget gate, input gate, and output gate). The cell state helps the network consider long-term dependencies. The cell state can be compared analogously to a conveyor belt: it moves straight down the entire network, with only a few minor linear interactions. In contrast, the hidden state can be compared analogously to a collection of buckets: the hidden state at time t is the set of values ​​that represent what the network "remembers" at that point in time. Because the hidden state is reset at the beginning of each sequence, it does not contain any information about previous sequences.

[0052] Regarding training, a model including one or more LSTM network units can be trained, for example, with a supervised learning approach. For example, a training dataset having a set of training sequences can be used. Each training sequence can have example signal data sequences. For example, in the present context, the training data can have a set of example IMU data signals. An optimization algorithm such as a gradient descent algorithm may be employed in combination with backpropagation through time to calculate the gradients needed during the optimization process. The goal is to change each weight of the LSTM network proportionally to the derivative of the error (at the output layer) with respect to the associated weight.

[0053] For more details regarding LSTM neural networks, see the publicly available document "Long short-term memory," by Hochreiter, Sepp, and Jurgen Schmidhuber, Neural Computation 9.8 (1997):1735-1780, which provides a detailed description of the structure and implementation of suitable LSTM networks. In particular, Chapters 2 and 3 of this document provide background on the special challenges that LSTM networks overcome compared to other types of networks. Chapter 4 provides a discussion of the general structure of LSTM networks. Chapter 5 presents a series of example implementations and related results.

[0054] Referring again to the exemplary neural network model of FIG. 1 , input data 102 to the model comprises real-time IMU data signals. The first element of the model comprises an encoder 104 having a single 1D convolution block or layer 122 followed by a batch normalization (“batch norm”) layer 124. During training, the encoder 102 can learn feature representations of the input sensor data 102. The feature representations are captured in feature maps. After training, the encoder operates to encode the input sensor data 102 in terms of the feature representations learned during training.

[0055] The output of the encoder 104 is fed to a single-layer long-short-term memory (LSTM) neural network unit 106. During training, the LSTM unit is able to model the time dependencies between activations of the feature maps.

[0056] The output of the LSTM block (i.e., the final hidden state) is input to the head of the classifier 108, which has a 1D convolutional layer 154 preceded by two 1D convolutional blocks 150, 152. For additional background, in neural networks, the term "hidden layer" is used to refer to a layer located between the input and output of an algorithm, where a function applies weights to the inputs and directs them as outputs via activation functions. In contrast, "hidden state," in the context of recurrent neural networks, refers to the set of variables used in the recurrent operation of the network. The hidden state is effectively the input to what is happening in the model at a given step and is calculated based on data from the previous time step. It can be understood as the result of one hidden layer being passed as input to the next hidden layer.

[0057] Each of the convolutional blocks 150, 152 includes a separate convolutional layer 132, 138, followed by a batch normalization layer 134, 140, followed by a rectification (ReLu) layer 136, 142. The final convolutional layer group 154 ​​includes a convolutional layer 144 combined with a batch normalization layer 146. Note that other configurations are possible, such as varying the number of layers in the encoder 104, LSTM 106, and classification module 108.

[0058] The model output 110 generated by the classifier module 108 using the output of the LSTM 106 comprises, for each time point of the real-time input data signal 102, a separate probability prediction for each of a plurality of location (sub)segments of the relevant anatomical structure, from which one of the location sub-segments can be identified that is most likely for the current location of the relevant device part.

[0059] By way of example, an exemplary implementation of a model for predicting the real-time position of a brush head within the oral cavity was created and trained. To this end, the classifier module 108 was configured to generate 12 (probability) prediction outputs, one for each of the 12 position sub-segments of the mouth. Figure 2 shows a schematic representation of the 12 position sub-segments of the mouth. The numbered layers correspond as follows: 0: Top right outer 1: Upper front outer 2: Upper left outerwear 3: Lower left outerwear 4: Lower front outer 5: Bottom right outerwear 6: Upper right inner 7: Upper front inner 8: Upper left inner 9: Lower left inner 10: Lower front inner 11: Lower right inner.

[0060] Using such a predictive model, localization accuracy will generally be lower at the beginning of a brushing session compared to later time periods in the session when the user has been brushing for a period of time and therefore has a greater amount of recent history from which predictions can be generated. The memory capabilities of the LSTM units described above allow the model to account for time dependencies in the input data as time passes in a personal care session.

[0061] To test this hypothesis, a baseline model, shown in Figure 1, was constructed and trained and tested using a holdout / test dataset consisting of IMU data signals from 1056 previously recorded brushing sessions.

[0062] The results are shown in the box plots in FIG.

[0063] A large variation in localization accuracy is observed at the beginning of a brushing session. Each boxplot shows the localization accuracy (y-axis) for the first n seconds from the start of the brushing segment (x-axis). The final boxplot represents the accuracy averaged over the entire brushing session.

[0064] As an example, we observe that the median accuracy for the first n=2 seconds is approximately 78%. Furthermore, the quartiles of the boxplot show a large spread around the mean, especially in the lower (i.e., first and second) quartiles. On the other hand, the boxplot of accuracy for longer session times (e.g., n=90 seconds) shows less variability in the lower quartiles and a slightly higher median accuracy of approximately 82%. This indicates that prediction accuracy improves as time elapses since the start of the session. After longer time periods, the model has access to more context about the behavioral history of the brushing session to estimate the currently brushed segment.

[0065] Therefore, we recognize that the explanation for the poor performance at the beginning of the session lies in the fact that the real-time localization model of Figure 1 does not utilize any prior knowledge about the actual starting position of the brush head within the oral cavity (i.e., oral cavity segment, in the case of the model of Figure 1). The same principle applies to any personal care device that is tracked relative to any relevant anatomical structure.

[0066] An embodiment of the present invention proposes to address the above identified problem by using an AI model configured to estimate the real-time current location of at least a part of a personal care device, said estimation being conditioned by information about the starting location of said at least a part of said personal care device.

[0067] According to a particular set of exemplary embodiments of the present invention, it is proposed to address the above-mentioned problems by modifying the basic model structure of the model of Figure 1 so that the LSTM module of the model is configured to receive as an additional input a representation of the starting position of the part of the personal care device being tracked. This can be provided as an input to the localization model at the start of the inference (i.e., at the start of the personal care session). As an example, when tracking the position of a brush head within the oral cavity, the starting position of the brush head (at the start of the session) can be encoded as one of the oral cavity segments (e.g., encoded as an integer value between 0 and 11) for which a prediction output is generated by the classifier module.

[0068] A particular set of embodiments of the present invention proposes using a recurrent artificial neural network including at least one LSTM unit to estimate the real-time current position of at least a portion of a personal care device, where said estimation is conditioned by a starting position of said at least a portion of said personal care device. However, more generally, any AI model trained to receive a starting position as input can be used.

[0069] 4 outlines in block diagram form the steps of an exemplary method 10 according to one or more embodiments. Before being further described in the form of exemplary embodiments, the steps will be summarized.

[0070] The method 10 is for real-time localization of parts of a personal care device during a personal care session.

[0071] The method 10 includes the step 12 of obtaining or receiving data indicative of a starting position for a portion of a personal care device for a personal care session.

[0072] The method 10 further comprises receiving 14 real-time inertial measurement unit (IMU) signal data during the personal care session.

[0073] The method 10 further includes retrieving 16 a real-time localization model from the data store. The localization model comprises an artificial intelligence (AI) model that is trained to receive a first input including data indicative of a starting location of the portion of the personal care device and a second input including real-time IMU signal data, and to generate as an output a real-time localization signal indicative of a real-time predicted location of the device portion.

[0074] The method 10 further comprises providing 18 the obtained device part starting positions as inputs to a localization model.

[0075] The method 10 further comprises providing 20, over the course of the personal care session, the received real-time IMU signal data for the personal care session as input to a localization model.

[0076] The method 10 further includes the step 22 of receiving an output real-time localization signal from the localization model.

[0077] The method 10 preferably further includes the step 24 of generating a data output based on the real-time localization signal.

[0078] As mentioned above, the method may also be embodied in the form of hardware, for example in the form of a processing device configured to perform the method according to any example or embodiment described herein or according to any claim of the present application.

[0079] To further aid in understanding, Figure 5 depicts a schematic diagram of an exemplary processing device 32 configured to perform methods according to one or more embodiments of the present invention. The processing device is shown in the context of a system 30 having the processing device. The processing device alone represents one aspect of the present invention. The system 30 is another aspect of the present invention. A provided system need not include all of the illustrated hardware elements; it may include only a subset of them.

[0080] The processing device 32 comprises one or more processors 36 configured to perform the method according to the above summary or according to any embodiment described herein or claimed in the present application. In the illustrated example, the processing device further comprises an input / output 34 or communication interface.

[0081] In the illustrated example of Figure 5, system 30 further includes a user interface 52. In some embodiments, the localization signal can be communicated to the user interface to convey localization information to the user. In some embodiments, an indication of the starting location can be input by the user at user interface 52, received by processing unit 32, and provided as an input to the localization model, as described below. In some embodiments, the user interface is included in a mobile computing device, such as a smartphone. In some embodiments, the user interface is included in a personal care device.

[0082] The system 30 in this example further includes the personal care device 54 itself. The personal care device 54 includes an inertial measurement unit (IMU) 56. The IMU can include one or more accelerometers for generating accelerometer data. The IMU can include at least one gyroscope for generating gyro data. In some embodiments, the IMU signal data output from the IMU can include accelerometer data and / or gyroscope data. In some embodiments, the IMU signal data output from the IMU can include 3D accelerometer and 3D gyro sensor readings. The IMU signal data 44 can be communicated from the IMU of the personal care device 54 to the processing unit 32 to provide as input to the localization model.

[0083] The system 30 in this example further comprises a data store 58 that stores a localization model 60. The method performed by the processing unit 32 may comprise retrieving the localization model 60 from the data store.

[0084] The present invention may also be embodied in software. Therefore, another aspect of the present invention is a computer program having computer program code configured, when executed on a processor, to cause the processor to perform a method according to any embodiment described herein or according to any claim of the present application.

[0085] Referring again to the system 30 of FIG. 5 , it should be noted that the system or processing unit 32 may include a memory 38 that stores computer program instructions for execution by one or more processors 36 of the processing unit to cause the one or more processors to perform a method according to any of the embodiments described in this disclosure.

[0086] Processing of sensor data 44 from the IMU 56 can be handled in multiple ways depending on the system architecture. For example, in some embodiments, the method of the present invention may be implemented by a software module. The software module may be implemented by a processing device included in a mobile computing device, a personal care device, or a separate processing device, or may be implemented by a remote server or cloud server. As an example, IMU sensor data may be synchronized to a smartphone app, and a software module integrated into the app may perform method 10 and estimate the brush head position using the sensor data. Alternatively, software for brush head localization may be embedded in the personal care device itself, for example, in the device handle. Again, the sensor data may be synchronized to a cloud infrastructure, and a software module for performing method 10 may also be executed as part of the cloud infrastructure. Output from this method may be communicated to the user and presented using a user interface, for example, by an app executed by the smartphone or by a user interface included in the personal care device.

[0087] One application area is an oral care area. In some embodiments, the personal care device 54 is an oral care device, and the personal care session is an oral cleaning session. At least a portion of the device being tracked can include, for example, a brush head. In further embodiments, the personal care device can be a skin care device, such as a light therapy device, and the personal care action is a skin care action, and the personal care session is a skin care session. At least a portion of the device can be a manipulable skin treatment portion, such as a light output area in the case of a light therapy device. In further embodiments, the personal care device is a grooming device, such as a shaver, and the personal care action is a grooming action (e.g., shaving), and the personal care session is a grooming session (e.g., shaving). At least a portion of the device being tracked can include, for example, a shaver head. In further embodiments, the personal care device is a muscle / joint care device (e.g., a massage device). In each case, the personal care device is a device for application to an area of ​​a user's body for a care function, whereby normal use of the device essentially involves moving the device around different locations on one or more areas / anatomical regions of the body.

[0088] To explain and illustrate the inventive concept, an example is presented below with reference to an oral care device in which the aforementioned personal care action is a cleaning action and the personal care session is an oral cleaning session. However, it will be recognized that the outlined principles are easily applicable to other types of personal care devices, such as, by way of example, skin care devices, e.g., phototherapy devices, grooming devices, e.g., shavers, or muscle / joint care devices (e.g., massage devices). In any personal care device, the device is typically designed to be applied to an area of ​​the user's body for a care function and can be moved around different locations. It can therefore be seen how the principles of the general inventive concept outlined above and described below can be used in any personal care device. Accordingly, references to an oral care device in the following embodiments can be replaced with references to any other personal care device without substantial changes to the inventive principles. Similarly, references to a cleaning action can be replaced with references to a personal care action, and references to a cleaning session can be replaced with references to a personal care session.

[0089] When the personal care device 54 is an oral care device, the oral care device may, in some embodiments, be a power toothbrush. The toothbrush may have a brush head carrying an array of cleaning elements, such as bristles.

[0090] The data processing flow according to one or more embodiments is shown in schematic form in Figure 6. As shown, a localization model 60 is configured to receive at least two data inputs. The first data input comprises data representing a starting position 64 for a portion of a personal care device for a personal care session. The second input comprises real-time inertial measurement unit (IMU) signal data 44 for the personal care session. The localization model 60 generates a real-time localization signal 68 as an output.

[0091] With respect to the localization model, as mentioned above, this comprises an artificial intelligence model. The AI ​​model may be an artificial neural network. In a preferred embodiment, the AI ​​model is a recurrent artificial neural network. The recurrent neural network may be an LSTM artificial neural network.

[0092] An exemplary implementation of a localization model according to one or more embodiments will now be described with reference to Figure 7. The model structure may be the same as the model of Figure 1 (already described above), except that the LSTM unit 106 is configured to receive an additional input indicating a starting position 64 of at least a portion of the personal care device to be localized and to utilize the input starting position in inferring the localization. In some embodiments, this may be achieved through learnable embeddings 162, as will now be described.

[0093] In particular, in some embodiments, the localization model comprises an LSTM artificial neural network, and the method comprises initializing cell states and hidden states of the LSTM artificial neural network based on a first input comprising data indicating a starting location of the portion of the personal care device. More particularly, in some embodiments, the localization model comprises at least one embedding layer configured to map input starting locations to at least one embedding vector, and configured to initialize cell states and hidden states based on the at least one embedding vector. Even more particularly, in some embodiments, the localization model comprises at least two embedding layers, one embedding layer for outputting an embedding vector for initializing the hidden states based on an input indicating the starting location, and one embedding layer for outputting an embedding vector for initializing the cell states based on an input indicating the starting location.

[0094] This is explained in more detail below.

[0095] In the context of neural networks, an embedding is a low-dimensional vector representation of the relationships present in high-dimensional input data. More specifically, an embedding is a mapping of discrete (categorical) variables to a vector of continuous numbers. In the context of neural networks, an embedding is a low-dimensional, learned continuous vector representation of a discrete variable. The distance between embedding vectors captures the similarity between different data points and can capture higher-level essential concepts in the original input. Neural network embeddings are actually an inherent generative by-product of a typical supervised training process. In particular, an embedding is a vector representation of the network's parameters (weights), which are adjusted during training to minimize loss for the supervised task. The resulting embedded vector is a representation of categories, with similar categories for the task being closer to each other. In this context, the task is localizing at least a portion of a personal care device. The categories can be, for example, different starting positions, encoded with one of a discrete set of integer values ​​corresponding to different position segments. Thus, in the context of the present invention, there will be one embedding vector for each possible starting position (e.g., there will be a discrete number of starting positions, encoded with integer numbers corresponding to particular segments of anatomy, etc.). In operation, the embedding layer provides the functionality to map an input indicator of a starting position (e.g., in the form of an integer value) to a corresponding one in a set of learned embedding vectors, where the corresponding learned embedding vector can be used as initialization for the hidden and / or cell states of the LSTM. As described below, there can be two embedding layers: one for outputting learned embedding vectors for initializing cell states, and one for outputting learned embedding vectors for initializing hidden states.

[0096] As mentioned above, recurrent neural networks operate by recursively processing a hidden state, which contains a set of variables and is passed as output from one step in the network's processing to the next. LSTMs also use cell states, which hold "memory" information that persists for a longer period of time than the hidden state. Each processing step in an LSTM functions by taking as input the hidden state and cell state output from the previous step and generating updated hidden and cell states.

[0097] Thus, a hidden state refers to the set of variables used within the recurrent operation of a neural network; it is effectively the input to what is happening in the model at a given time step, calculated based on data from previous time steps. It can be understood as the result of one hidden layer being passed as input to the next hidden layer.

[0098] The cell state effectively acts as a memory, carrying information throughout the processing chain and is updated at each processing step using the various gates of the LSTM cell (forget gate, input gate and output gate).

[0099] In the context of some embodiments of the present invention, each embedding vector therefore has the same size / dimension as the hidden state or cell state, respectively.

[0100] Given an input starting position, e.g., encoded as an integer, the model under training learns a latent embedding vector in an end-to-end manner while training the remaining model parameters. In particular, at the beginning of training, the associated starting position for a given training data signal is input (e.g., in the form of an integer value) and passed as input to two different embedding layers: one embedding layer for the hidden state and one embedding layer for the cell state. To explain, an LSTM generally has two states internally: a hidden state and a cell state. Therefore, we propose to use two embedding layers to learn and condition each of these states based on the starting position.

[0101] Each embedding layer maps a starting location (e.g., an integer) to a latent vector of the same size as the LSTM's hidden state, or cell state. This latent embedding is used by the LSTM model during both training and inference to predict real-time localizations in the form of class labels associated with one of a discrete set of possible locations.

[0102] In operation, for inference, the learned embedding effectively behaves as a lookup table: input starting positions are mapped to corresponding embedding vectors for the hidden and cell states, respectively. The embedding vectors are used to specify the initialization of the hidden and cell states. Specifically, this means that the starting states of the hidden states are set to the values ​​specified in the hidden state embedding vectors output from the respective embedding layers, and the starting states of the cell states are set to the values ​​specified in the cell state embedding vectors output from the respective embedding layers.

[0103] Thus, an embedding is learned during model training (the embedding layer is part of the model) that allows mapping an input starting position (e.g., represented as an integer value) to a real-valued vector output. This output is used as initialization for the hidden and cell states of the LSTM model. Otherwise, the states are initialized to zero at the start of the inference procedure. The hidden and cell states are updated recursively during the operation of the model as part of the network operation.

[0104] Note that if no starting positions are provided, there are two options for how to initialize the hidden and cell states: for example, a default initialization corresponding to an "average" starting position based on global statistics (e.g., the most frequent starting position in the general population) can be used, or the hidden and cell states of the LSTM can be initialized with zeros.

[0105] The high-level model structure is shown in FIG.

[0106] The structure of the model may be the same as in Figure 1 (described in detail above), except for the additional use of learned embeddings 162 corresponding to different possible input start locations 64 of at least a portion of the personal care device. The embeddings corresponding to the input start locations 64 are used to initialize the hidden states and cell states of the LSTM units 106 of the neural network at the beginning of a personal care session.

[0107] Therefore, embodiments of the present invention propose to integrate starting positions to develop conditional variants of the real-time segmentation mode with the aim of improving the generalization of the model at the beginning of a session. For this purpose, it is proposed to have a learnable embedding vector 162 for each possible starting position (i.e., output class) that can be used as an initialization for the hidden and cell states of the LSTM.

[0108] This minimal and effective approach significantly improves the generalization of the model without any overhead.

[0109] Note that any of the implementation details described in connection with the model of Figure 1 can also be applied to the model of Figure 7, which is similar to the model of Figure 1 except that it includes one or more additional buried layers.

[0110] The above represents just one possible approach to providing a model in which starting positions can be used as input. Other approaches to integrating starting positions into deep neural network models are also possible.

[0111] According to one alternative, in an LSTM-based model, instead of having separate learnable embedding layers for the hidden state and the cell state, a single embedding layer can be shared to further reduce model size. The dimensions of these states are the same. Therefore, the output of a single embedding layer can be used as the initialization for both states. However, as explained above, the hidden state and the cell state are intended to learn different things and control different aspects of the LSTM cell, and therefore, using two embedding layers may be preferable.

[0112] Additionally or alternatively, it is also an option to condition the output of the LSTM unit 106 by summing with a latent vector, as opposed to using the latent embedding vector as the initialization of the LSTM's state. In this case, the embedding layer provides an output with the same dimensionality as the LSTM's output. For example, when specifying the neural network architecture at the beginning of training, one might set the output size of the LSTM to 128 and the output size of the embedding layer to 128 as well. Here, we can simply sum the outputs of the LSTM and the embedding layer with an element-wise sum over the vector. In a deep learning library like PyTorch, this can be achieved with the torch.add function.

[0113] Additionally or alternatively, feature-wise linear modulation (FiLM) can be incorporated at the level of the convolutional encoder 104 to condition features in earlier layers of the model. Regarding feature-wise linear modulation, see the paper by Perez, Ethan et al., "Visual reasoning with a general conditioning layer," Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32, No. 1, 2018.

[0114] Formally, feature-wise linear modulation is a general conditioning layer that modulates input features. A conditioning layer is a neural network that takes as input the features of another layer (e.g., an encoder) and the output of a previous FiLM layer (if available), and outputs a set of modulation parameters. These modulation parameters are used to modulate the input features (by performing element-wise multiplication and addition) to generate new feature maps. FiLM layers can be stacked on top of other layers to form deep FiLM networks.

[0115] The FiLM layer is useful because it allows the network to learn how to modulate features of the input signal to produce a desired output. For example, if the goal is to incorporate the starting location to increase the likelihood that the network will correctly predict the real-time localization position toward the start of a personal care session, the FiLM layer can learn to modulate the features of the previous layer, so that the starting location is incorporated into the features to increase the likelihood of the desired output.

[0116] At the implementation level, the output of the convolutional encoder 104 (as shown in Figure 1 or Figure 7) is passed as input to the FiLM layer, with the output of the embedding layer used as a conditioning mechanism. The FiLM layer performs modulation of the encoder features based on the embedding vector representing the starting position. Finally, the output is passed to another LSTM.

[0117] Note that a FiLM layer works like any other layer in a deep model, having a single linear layer that transforms the condition input (in our case the output of the embedding layer) which is used to update the output of the encoder via element-wise multiplication and addition operations.

[0118] It should be noted that the above approaches can be combined to further improve the performance of the model.

[0119] As mentioned above, in an advantageous embodiment, the starting position is encoded in the form of an integer value having a predetermined range of possible values.

[0120] Different values ​​of the integer value can correspond to different body segment positions, for example different oral segment positions.

[0121] FIG. 8 shows the model performance of the proposed invention using the same test dataset used to test the performance of the basic model outlined in FIG. 1 and whose results are shown in FIG. 3. To test the model, the model was provided with the session's real-time IMU signal data and the true start position of the session obtained from the reference data as inputs for each cleaning session. Comparing the results of the basic model in FIG. 3 with those of the proposed model in FIG. 8, it is clear that the proposed model significantly improves both the average localization accuracy and the spread in localization accuracy relative to the average value. In particular, the time periods close to the start of the personal care session (e.g., 2 seconds, 5 seconds, and 10 seconds from the start) show very significant improvements in average accuracy and accuracy spread. This therefore indicates that additional use of the start position as a conditional initialization parameter in the model improves localization accuracy toward the start of the personal care session.

[0122] Next, some optional implementation features of the method 10 in operation are described.

[0123] To implement the method, a personal care device, such as an electric toothbrush, can be equipped with electronics that allow synchronization with a software module, such as an app, running on a computing device, such as a mobile computing device, or running on the cloud. The personal care device can have an IMU sensor, such as those described above, consisting of a 3-axis accelerometer and a 3-axis gyroscope.

[0124] In some embodiments, the user may be prompted by the app to input a preferred or planned start position for the personal care device, e.g., a planned segment in the oral cavity where brushing is to begin. In other words, in some embodiments, obtaining data indicative of the start position comprises receiving user input from a user interface indicative of the planned start position. The start position can be added to a configuration data set. Optionally, the configuration data set can additionally be transferred to the personal care device, e.g., via a wireless connection such as Bluetooth.

[0125] For example, a user interface can be provided that allows user input of a planned starting location.

[0126] Optionally, in some embodiments, the software module may include one or more algorithms configured to automatically determine a predicted start position based on the user's personal care device usage history, e.g., brushing history. In other words, obtaining data indicative of the start position may include accessing a data store recording the user's past personal care data and determining a predicted start position based on processing the past personal care data. Once the start position is determined, the user interface may be further controlled to provide a prompt that allows the user to confirm or adjust the automatically determined start position. In other words, the method may include generating a control signal to control the user interface to generate a user-perceptible prompt requesting confirmation of the predicted start position.

[0127] A common way to realize a user interface (UI) is to use an app running on a mobile computing device, such as a smartphone. In other words, the user interface may be a user interface included in the mobile computing device. However, this is by no means the only way to realize a UI. The UI can also be realized in the cloud, or even in the personal care device, depending on the processing power and connectivity.

[0128] With regard to the execution of the pre-trained localization model, which may be realized, for example, by a processing module included in a mobile computing device, by a processing module in a cloud computing architecture, or by a processing module in a personal care device, the model outputs a real-time localization signal indicative of the real-time position of the personal care device, for example, the position of a brush head within the oral cavity.

[0129] At the start of a brushing session, the user can be prompted by a prompt generated by the user interface to place at least a portion of the personal care device in a designated starting position. The system can be configured so that the start of the session is triggered by a user-initiated signal generated by actuation of a user control element, such as the user pressing a power button on the personal care device. At the start of the session, IMU signal data can be sampled from the sensor unit and provided as input to a localization model in real time, along with the starting position of the device. The model initializes using the starting position information, after which real-time localization estimation begins.

[0130] Optionally, a further software module may be included as part of the system that receives as input the current predicted position of at least a portion of the personal care device and calculates a real-time coverage estimate for the relevant anatomical structure or relevant region of the anatomical structure. For example, in the context of an oral care device, such a software module may be configured to update the current coverage estimate (e.g., brushing time spent) for the particular oral cavity segment currently being cleaned.

[0131] In some embodiments, the personal care device can have one or more indicator elements, such as an indicator light, to communicate real-time coverage of the current anatomical region, e.g., brushing coverage of the oral cavity segment currently being brushed. For example, the indicator light may comprise a color-coded light emitted by a light ring integrated into the surface of the housing of the personal care device.

[0132] In some embodiments, instead of asking the user to provide a preferred starting position, a dedicated software module can alternatively be utilized to predict / obtain the most likely starting position by analyzing starting position estimates obtained from previous sessions performed by the user.

[0133] In other words, as a more general principle, in some embodiments, obtaining data indicative of a start location can include accessing a data store that records the user's past personal care data and determining a predicted start location based on processing the past personal care data.

[0134] In this case, as an example, the predicted start position is obtained using a post-brushing localization model (also known as an offline model), which, for example, runs in a cloud environment and has access to raw IMU data from previous sessions stored in the same cloud infrastructure. The offline prediction model may generally have higher accuracy compared to a real-time prediction configuration because the post-brushing prediction model is not bound by any causal dependencies and can consider both forward and backward dependencies in time. The offline prediction model is provided as having substantially the same structure and operation as the model in FIG. 7, except for the fact that the LSTM layer is modified to be bidirectional, allowing for both forward and backward inference. This allows the model to perform backward inference to determine a start position prediction. Alternatively, the offline prediction model may have a different architecture than that in FIG. 7. One example is a UNet-based localization model. The offline model is configured to process IMU data from multiple personal care sessions to obtain the predicted "most frequent" start position. Such an approach can result in a more accurate prediction of the most likely start position. This is because the entire IMU dataset of one or more personal care sessions can be processed offline after the relevant sessions have finished.

[0135] In other words, in some embodiments, the past personal care data can include IMU signal data of past personal care sessions, and predicting the start position of the personal care session includes processing the past IMU signal data to estimate past start positions, and predicting the start position based on the estimated past start positions.

[0136] In a simpler implementation, the user's past cleaning personal care data may include past start position data, and predicting the start position comprises processing the past start position data.

[0137] In some embodiments, the method may confirm the predicted start location to a user, i.e., the method further comprises generating a control signal for controlling a user interface to generate a user-perceptible prompt requesting confirmation of the predicted start location.

[0138] In some embodiments, once the most likely starting segment for the next brushing session is obtained from the user's past session data, the software module can compare it to the currently configured starting segment recorded in the configuration data set, and if there is a difference, the software app may generate a message dialog in the user interface asking the user whether they want to update the currently configured starting position.

[0139] In other words, as a more general principle, in some embodiments, the method includes the steps of additionally receiving a user setting of a start location, comparing the user setting of the start location with the predicted start location, and, based on detecting a difference therebetween, generating a control signal for controlling a user interface to generate a user-perceptible prompt requesting confirmation of the predicted start location.

[0140] According to at least one set of embodiments, providing a starting location can be optional. This means that the system includes two localization models dedicated to two specific tasks. A first model may be trained end-to-end specifically for localization when a starting location is not provided or is not yet known. A second model may be trained end-to-end specifically for localization when a planned starting location is provided by the user or when a reliable user-specific starting location is inferred from the user's past session data. Based on whether a preferred starting point has been provided or becomes available, the solution selects the appropriate model for localization inference.

[0141] In other words, in some embodiments, the method may further comprise a preliminary mode selection step including selecting a first or second mode. One mode is configured for localization using an input start position, and the other mode is configured for localization without an input start position. For example, based on the selection of the first mode, the steps of the method according to the previously described embodiment are applied. For example, this may be a model according to FIG. 7. Based on the selection of the second mode, a second localization model is obtained, configured to perform real-time localization without an input indicating a start position, and real-time IMU signal data is provided as input to the second localization model. For example, this may be a model according to FIG. 1.

[0142] Another option is to provide a system where providing a starting location is optional, but the system has a single localization model that can handle two situations: the first is when a preferred starting location has not been provided by the user or is not yet known, and the second is when a preferred starting location is provided or is inferred from the user's past session data.

[0143] In other words, in some embodiments, a single localization model may be initializeable in two states, where the first state is configured for the model to generate localization data based on using an input indicating a starting position, and the second state is configured for the model to generate localization data without using an input indicating a starting position. The method may further include a preliminary mode selection step including selecting a first mode or a second mode, where based on the selection of the first mode, the localization model is used in the first initialization state, and based on the selection of the second mode, the localization model is used in the second initialization state.

[0144] In this approach, the model is initialized differently depending on whether a starting position is provided or not. For example, when the starting position is known, the model is initialized by determining the hidden and cell states of the LSTM and selecting a specific embedding value associated with the provided starting position, obtained from training. Alternatively, when the starting position is unknown, the model is initialized by assigning all zeros to the hidden and cell state vectors of the LSTM. Note that such a model that can handle both cases can be trained end-to-end.

[0145] The above-described embodiments of the present invention employ a processing device. A processing device can generally have a single processor or multiple processors. It may be located in a single containing device, structure, or unit, or may be distributed among several different devices, structures, or units. Thus, a reference to a processing device being adapted or configured to perform a particular step or task may correspond to that step or task being performed by any one or more of several processing elements, alone or in combination. Those skilled in the art will understand how such a distributed processing device can be implemented. A processing device may include a communication module or input / output to receive data and output data to additional elements.

[0146] The one or more processors of the processing unit can be implemented in several ways using software and / or hardware to perform the various functions required. A processor typically uses one or more microprocessors, which can be programmed using software (e.g., microcode) to perform the required functions. A processor can also be implemented as a combination of dedicated hardware to perform some functions and one or more programmed microprocessors and associated circuitry to perform other functions.

[0147] Examples of circuitry that may be employed in various embodiments of the present application include, but are not limited to, conventional microprocessors, application specific integrated circuits (ASICs), and field programmable gate arrays (FPGAs).

[0148] In various implementations, a processor may be associated with one or more storage media, such as volatile and non-volatile computer memory, including RAM, PROM, EPROM, and EEPROM. The storage media may be encoded with one or more programs that, when executed by the one or more processors and / or controllers, perform the required functions. The various storage media may be fixed within the processor or controller, or may be transportable such that one or more programs stored thereon can be loaded into the processor.

[0149] Variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the figures, the disclosure and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality.

[0150] A single processor or other unit may fulfill the functions of several items recited in the claims.

[0151] The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

[0152] The computer program may be stored / distributed on a suitable medium, such as an optical storage medium or a solid-state medium, supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless communication systems.

[0153] It should be noted that when the term "adapted to" is used in the claims or specification, the term "adapted to" is intended to be equivalent to the term "configured to."

[0154] Any reference signs in the claims should not be construed as limiting the scope of the invention.

Claims

1. 1. A method for real-time localization of at least a portion of a personal care device during a personal care session, comprising: obtaining data indicative of a starting position of at least a portion of the personal care device for the personal care session; receiving real-time inertial measurement unit (IMU) signal data during the personal care session; retrieving a real-time localization model from a data store, the localization model comprising an artificial intelligence (AI) model, the localization model receiving a first input comprising data indicative of a starting location of the at least one portion of the personal care device and a second input comprising real-time IMU signal data, and generating as an output a real-time localization signal indicative of a real-time predicted location of the portion of the device; providing the obtained starting position of the device part as an input to the localization model; providing the received real-time IMU signal data relating to the personal care session as input to the localization model over the course of the personal care session; receiving an output real-time localization signal from the localization model; and generating a data output based on the real-time localization signal.

2. The method of claim 1 , wherein the AI ​​model is an artificial neural network, preferably a recurrent neural network.

3. 3. The method of claim 1 or 2, wherein the AI ​​model is a long short-term memory artificial neural network.

4. 4. The method of claim 1, wherein the personal care device is an oral care device, and the method is for real-time localization of a head of the oral care device within a user's oral cavity during a cleaning session.

5. 5. The method of claim 1, wherein obtaining data indicative of the start location comprises receiving user input indicative of an intended start location from a user interface, optionally the user interface being a user interface included in a mobile computing device.

6. 5. The method of claim 1, wherein obtaining data indicative of the start location comprises accessing a data store that records historical personal care data of the user; and determining a predicted start location based on processing the historical personal care data.

7. the user's historical personal care data includes historical start position data, and predicting the start position comprises processing the historical start position data; and / or 7. The method of claim 6, wherein the historical personal care data includes IMU signal data of past personal care sessions, and predicting a start location of the personal care session comprises processing the historical IMU signal data to estimate past start locations and predicting the start location based on the estimated past start locations.

8. The method further comprises: receiving a user setting of the starting location; comparing the user setting of the starting position with the predicted starting position; and generating a control signal based on detecting a difference between the two to control a user interface to generate a user-perceptible prompt requesting confirmation of the predicted starting position.

9. the method further comprising a preliminary mode selection step including selecting a first or second mode; Based on the selection of the first mode, the steps of the method according to any one of claims 1 to 8 are carried out, 9. The method of claim 1, wherein based on the selection of the second mode, a second localization model is obtained that performs real-time localization without an input indicating a starting position, and the real-time IMU signal data is provided as an input to the second localization model.

10. the localization model is initializable in two states, in a first state the localization model generates localization data based on using an input indicating a starting position, and in a second state the localization model generates localization data without using an input indicating a starting position; the method further comprising a preliminary mode selection step including selecting a first or second mode; 9. The method of claim 1, wherein, based on the selection of the first mode, the localization model is used in the first initialization state, and based on the selection of the second mode, the localization model is used in the second initialization state.

11. 11. The method of claim 1, wherein the starting position is coded in the form of an integer value having a predetermined range of possible values, optionally different values ​​of the integer value corresponding to different body segment positions, e.g. different oral segment positions.

12. 12. The method of any one of claims 1 to 11, wherein the AI ​​model comprises a long short-term memory (LSTM) artificial neural network, the method comprising initializing cell states and hidden states of the LSTM artificial neural network based on a first input comprising data indicative of a starting position of the portion of the personal care device.

13. 13. The method of claim 12, wherein the localization model comprises at least one embedding layer that maps a first input indicating a starting position to at least one embedding vector, and initializes cell states and hidden states of the LSTM artificial neural network based on the at least one embedding vector.

14. 14. The method of claim 13, wherein the localization model has at least two embedding layers: one embedding layer for outputting an embedding vector for initializing the hidden state based on an input indicating a starting location, and one embedding layer for outputting an embedding vector for initializing the cell state based on an input indicating a starting location.

15. A computer program having a computer program code which, when executed on a processor, causes the processor to perform the method according to any one of claims 1 to 17.

16. 1. A processing device for use in real-time localization of at least a portion of a personal care device during a personal care session, comprising: Input / output; and one or more processors for executing a method, said method comprising: obtaining data indicative of a starting position of at least a portion of the personal care device for a personal care session; receiving real-time inertial measurement unit (IMU) signal data at said input / output during said personal care session; retrieving a real-time localization model from a data store, the localization model comprising an artificial intelligence (AI) model, the localization model being trained to receive a first input comprising data indicative of a starting location of the portion of a personal care device and a second input comprising real-time IMU signal data, and to generate as an output a real-time localization signal indicative of a real-time predicted location of the portion of the device; providing the obtained head starting position as an input to the localization model; providing received real-time IMU signal data relating to the personal care session as input to the localization model over the course of the personal care session; receiving an output real-time localization signal from the localization model; generating a data output based on said real-time localization signal and optionally coupling said data output to said input / output.

17. 1. A system comprising: a personal care device including an integrated IMU for generating IMU signal data during a personal care session, the personal care device including a wireless communication module for transmitting the IMU signal; and a processing device according to claim 16 for receiving the IMU signal data transmitted by the personal care device. Optionally, the system comprises a mobile communications device including a user interface.

Images