APPARATUS AND METHOD FOR PERFORMING LOCALIZATION OF MOBILE TREATMENT DEVICES - Patent application
Inertial sensors and neural networks enhance the localization and classification of mobile treatment devices by accurately mapping user-specific motion patterns, addressing existing localization and classification challenges.
Patent Information
- Application Number
- JP2022158618
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2018-02-19
- Filing Date
- 2022-09-30
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2039-02-15
AI Technical Summary
Existing localization and classification methods for mobile treatment devices, such as hairbrushes and vacuum cleaners, face challenges like coarse localization, GPS dependence on outdoor conditions, and user-specific motion classification issues, leading to inaccuracies and lack of personalization.
The use of inertial sensors and neural networks, specifically recurrent neural networks like GRU and LSTM, to classify motion patterns and accurately localize mobile treatment devices relative to target surfaces, while also personalizing the device operation based on individual user movements.
Enables precise localization and classification of mobile treatment device movements, adapting to individual user styles, improving accuracy and personalization, and enhancing user experience.
Smart Images

Figure 0007593974000002 
Figure 0007593974000003 
Figure 0007593974000004
Abstract
Description
[Technical field]
[0001] SUMMARY OF THE DISCLOSURE Embodiments of the present invention relate to an apparatus for performing localization of a mobile treatment device, a method for performing localization of a mobile treatment device, and a system including the apparatus and the mobile treatment device. [Background technology]
[0002] The mobile treatment device may be used to treat a surface, etc. For example, the mobile treatment device may be associated with a personal implement such as a hairbrush, razor, groomer, toothbrush, etc. In these examples, the surface to be treated may be the body, or at least a particular portion or zone of said body.
[0003] Other examples of mobile treatment devices can include household appliances such as brooms, mops, scrubbing brushes, etc. In these examples, the surface to be treated can be a floor, or at least a particular portion or zone of said floor.
[0004] In some applications, it may be useful to know the current location of the mobile treatment device. In some applications, it may additionally or alternatively be useful to classify the movement of the mobile treatment device, particularly in the case of personal appliances.
[0005] Nowadays, imaging techniques can be used to locate a mobile treatment device relative to a target surface, for example by a camera capturing the target surface. It may also be known to use sensors such as GPS sensors to locate a mobile treatment device. The imaging techniques described above can also be used to image the movement of a mobile treatment device and classify the captured movement.
[0006] These general devices and methods may work well for rough localization and classification. However, there may be some drawbacks. For example, GPS sensors may work well enough only in outdoor conditions. The view of the camera capturing the target surface may sometimes be obstructed by the mobile processing device itself. Furthermore, even if different users use mobile processing devices, such as personal appliances, the output of the above-mentioned devices and methods is always the same for each user, even though each user may have individual style and preferences of how to use the mobile processing device. Summary of the Invention [Problem to be solved by the invention]
[0007] It would therefore be desirable to provide apparatus and methods that allow for accurate localization of a mobile treatment device and / or accurate classification of the motion of the mobile treatment device without the drawbacks mentioned above. Furthermore, it would be desirable to personalize these apparatus and methods by providing different individuals with individually trained techniques. [Means for solving the problem]
[0008] According to one aspect, there is provided an apparatus for performing localization of a mobile treatment device relative to a target surface, the mobile treatment device comprising at least one inertial sensor and configured to treat the target surface, the apparatus comprising: a motion pattern classification device configured to discriminate between two or more motion classes contained in a set of motion classes for the mobile treatment device; and an interface (16) for providing at least one inertial sensor data from the inertial sensor to the motion pattern classification device, the at least one inertial sensor data representing movement of the mobile treatment device, the motion pattern classification device comprising at least one neural network configured to receive the at least one inertial sensor data and classify the at least one inertial sensor data against a motion class included in the set of motion classes, each of the motion classes being associated with one or more different zones relative to the target surface, whereby classification of the at least one inertial sensor data against a motion class indicates an estimate of a position of the mobile treatment device relative to the one or more zones of the target surface.
[0009] According to one aspect, there is provided a method for performing localization of a mobile treatment device relative to a target surface, the mobile treatment device comprising at least one inertial sensor and configured to treat the target surface, the method including: providing two or more motion classes included in a set of motion classes for the mobile treatment device; receiving at least one inertial sensor data from the inertial sensor, the at least one inertial sensor data representing a movement of the mobile treatment device; receiving and processing the at least one inertial sensor data by a neural network; and classifying the at least one inertial sensor data for the at least one motion class included in the set of motion classes, each of the motion classes being associated with one or more different zones of the target surface, whereby classification of the at least one inertial sensor data by the at least one motion class indicates an estimate of a location of a location of the mobile treatment device relative to the one or more zones of the target surface.
[0010] According to one aspect, there is provided a computer readable digital storage medium having a computer program stored thereon, the computer program having program code for performing the above-described method for performing localization of a mobile treatment device relative to a target surface, when running on a computer.
[0011] According to one aspect, there is provided a system comprising a mobile treatment device and an apparatus as previously described. [Brief description of the drawings]
[0012] In the following, embodiments of the invention will be described in more detail with reference to the drawings. [Figure 1] 1 shows a schematic block diagram of an exemplary apparatus according to the present disclosure. [Diagram 2] 1 illustrates an example of a target surface to be treated with a mobile treatment device. [Diagram 3] 13A-13C show further examples of target surfaces to be treated with a mobile treatment device. [Figure 4] FIG. 1 shows a schematic block diagram of a recurrent module (or unit or block or cell) that may be used in a recurrent neural network. [Diagram 5] FIG. 1 shows a schematic block diagram of a gated recurrent unit (GRU) module that may be used in a recurrent neural network. [Figure 6A] 1 shows a schematic block diagram of a recurrent LSTM module (or unit) that may be used in a recurrent neural network. [Figure 6B] FIG. 1 shows a schematic block diagram of an unrolled LSTM module-based recurrent neural network with one layer of LSTM modules at different time instances. [Figure 7]1 shows a schematic block diagram of an unrolled LSTM module-based recurrent neural network having two stacked layers of LSTM modules (i.e., deep LSTMs) at different time instances. [Figure 8] 1 shows a block diagram of the method of the present invention according to one embodiment. [Figure 9] 1 shows a schematic block diagram of an apparatus of the present invention, according to one embodiment; [Figure 10] 4 shows a schematic block diagram of an apparatus of the present invention according to a further embodiment; [Figure 11] 4 shows a schematic block diagram of an apparatus of the present invention according to a further embodiment; [Figure 12] 1 shows a block diagram of the method of the present invention according to one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0013] Equivalent or equivalent elements or elements having equivalent or equivalent functions are indicated in the following description with equivalent or equivalent reference signs.
[0014] In the following, reference is made to personal and / or household appliances as non-limiting examples of mobile treatment devices. However, these types of appliances are mentioned only as non-limiting examples to describe embodiments and examples of the present invention. Thus, the present invention is not limited to only these mentioned types of appliances.
[0015] Furthermore, the order of any method steps of the methods may be described as a non-limiting example only, and thus any method steps described herein may also be performed in any other order other than as described.
[0016] It will be appreciated that although some aspects are described in terms of an apparatus or device, these aspects also represent a description of a corresponding method, where a block or device corresponds to a method step or feature of a method step, and similarly, aspects described in terms of a method or method step also represent a description of a corresponding block or item or feature of a corresponding apparatus or device.
[0017] A first aspect of the invention relates to an apparatus configured to localize a mobile treatment device, in particular to localize a mobile treatment device with respect to a target surface to be treated with said treatment device.
[0018] Examples of such mobile treatment devices can be, for example, personal appliances, which can be, for example, hairbrushes, razors, shavers, epilators, beauty devices, groomers, toothbrushes (electric or manual toothbrushes), etc. In these examples, the surface to be treated can be the body, or at least a specific part or zone of said body.
[0019] Other examples of mobile treatment devices may relate to household appliances, such as brooms, mops, scrubbing brushes, etc. In these examples, the surface to be treated may be a floor or a wall or a ceiling, or at least a specific portion or zone of said floor, wall or ceiling.
[0020] FIG. 1 shows an apparatus 10 according to an embodiment of the present invention. Furthermore, a mobile treatment device 11 is depicted. The mobile treatment device 11 and the apparatus 10 together form a system. The mobile treatment device 11 may comprise at least one inertial sensor 13. Examples of the inertial sensor 13 are motion sensors (accelerometers providing linear acceleration), rotation rate sensors (gyroscopes providing angular velocity), and orientation sensors (magnetometers), such that the location, orientation, velocity, and acceleration of the mobile treatment device 11 can be determined directly and indirectly. Without being limited thereto, the inertial sensor may be a 3-axis inertial sensor, a 6-axis inertial sensor, or a 9-axis inertial sensor. The inertial sensor may in particular be realized as a MEMS sensor. Furthermore, the mobile treatment device 11 may be configured to treat a target surface 12.
[0021] As can be seen, the mobile treatment device 11 may be used in a specific location or position relative to the target surface 12, for example, in, on, or next to the target surface 12. The target surface 12 itself may be divided into one or more zones 211, 212, 213, ..., 214, ... m The zones 211, 212, 213, ..., 21 m may relate to the position of the mobile treatment device 11 outside the target surface, as will be explained in more detail further below. The mobile treatment device 11 may be moved relative to the target surface 12. It is a basic concept behind the apparatus and systems described herein that the movement of the mobile treatment device 11 depends on the zone in which it is used, and thus the zone in which it is used may be identified by classifying the movement, which is represented at least by inertial sensor data, which may be augmented by further sensor data, e.g. pressure sensor data.
[0022] The apparatus 10 of the present invention depicted in FIG. 1 is configured to perform localization of a mobile treatment device 11 relative to a target surface 12 .
[0023] In simple terms, the basic concept of this description is to use a neural network (NN) such as a recurrent neural network (RNN), in particular a Gated Recurrent Unit (GRU) RNN, a LSTM RNN (short LSTM), or a deep GRU RNN / LSTM, or even a deep bidirectional GRU RNN / LSTM, to classify (based on data from at least one inertial sensor) a temporal sequence of input vectors to learned motion classes, where the elements of the NN's output vector represent probabilities for each of the motion classes, and then map the sequence of the NN's output vector to one of a plurality of zones associated with the use of the mobile treatment device and output this zone. Instead of mapping a sequence of output vectors to zones, the output vectors can be mapped to more general outputs, such as "good" movements versus "bad" movements, or "correct" movements versus "incorrect" movements, or movements of a "first user" versus movements of a "second user", etc., meaning that classification occurs for movement classes representing these good or bad movements, etc. This is explained in more detail below.
[0024] The apparatus 10 may include a motion pattern classification device 14. The motion pattern classification device 14 classifies two or more motion classes 151, 152, 153, ..., 154 of the set of motion classes 15 of the mobile treatment device 11. k In other words, the mobile treatment device 11 may be moved in different linear and / or rotational directions, for example, by a user using said mobile treatment device 11. Thus, each movement of the mobile treatment device 11 may represent a respective or individual movement pattern. The movement pattern classification device 14 may include a set of different movement classes 15. The set of movement classes 15 may be classified into two or more movement classes 151, 152, 153, ..., 15 k The movement pattern classification device 14 may include these two or more movement classes 151, 152, 153, ..., 15k That is, the motion pattern classification device 14 may be configured to distinguish a first motion class 151 from a second motion class 152. Here, when it is said that the motion pattern classification device is configured to discriminate between the motion classes, it means that the motion pattern classification device has been trained to classify an input motion pattern into at least two motion classes.
[0025] The motion of the mobile treatment device 11 may be detected by at least one inertial sensor 13 included in the mobile treatment device 11. The inertial sensor 13 is an inertial-based sensor and may include at least one of an accelerometer, a gyroscope, and a magnetometer. The inertial sensor 13 may provide sensor data representing at least one of a linear velocity, an angular velocity, a linear acceleration, an angular acceleration, and gravity. The inertial sensor 13 may be part of an inertial measurement unit that includes one or more inertial sensors.
[0026] The device 10 receives at least one inertial sensor data 171, 172, 173, ..., 17 from the inertial sensor 13. n , and at least one inertial sensor data 171, 172, 173, ..., 17 n The motion pattern classification device 14 may include an interface 16 for providing at least one inertial sensor data 171, 172, 173, ..., 17 n represents the motion of the mobile treatment device 11. In other words, when the mobile treatment device 11 moves, the inertial sensor 13 detects this motion and generates at least one inertial sensor data 171, 172, 173, ..., 17 n Therefore, at least one inertial sensor data 171, 172, 173, ..., 17 n represents the respective motion of the moved treatment device 11. nmay be sampled continuously in time from the inertial sensor 13. A single sample of the inertial sensor data for one given time instant may be input to a motion pattern classification device to classify at least two motion classes 151, 152, 153, ..., 15 k The interface may in particular be capable of inputting a temporal sequence of inertial sensor data to the movement pattern classification device 13, although this does not exclude classifying movement patterns relative to the inertial sensor data, thereby allowing the temporal relationships of the inertial sensor data to be utilized in the classification process.
[0027] According to the present disclosure, the motion pattern classification device 14 may include a neural network 18. The neural network 18 may be a recurrent neural network or a deep neural network. The neural network 18 is configured to process the motion pattern classification data 171, 172, 173, ..., 174, based on at least one inertial sensor ... n , 151, 152, 153, ..., 15 k At least one inertial sensor data 171, 172, 173, ..., 17 n (The number of inertial sensor data n is independent of the number of motion classes k, i.e., 25 input vectors can be provided and 12 motion classes are used). This classification is represented in FIG. 1 by dashed and solid arrows 191, 192, 193, ..., 194, ... k The solid arrow 193 indicates that the neural network 18 is deriving the inertial sensor data 171, 172, 173, ..., 17 nIt can be exemplarily shown that the input motion pattern of the same time instant is successfully classified into the third motion class 153. It is understood that the output of the NN typically has the same sequence length as the input, i.e., if 10 input vectors are provided as an input sequence, 10 output vectors are provided as an output sequence (i.e., the RNN unit always generates an output vector for each time instant), and if only one input vector is provided, only one output vector is provided. Typically, each output vector of the NN includes k elements associated with k motion classes, and each element of the output vector provides the probability that the input motion pattern of the same time instant is associated with each corresponding motion class.
[0028] Different exercise classes 151, 152, 153, …, 15 k are exemplarily symbolized by different geometric shapes (circle, rectangle, triangle, star) purely for illustrative purposes.
[0029] According to the basic principles, the exercise classes 151, 152, 153, …, 15 k , 211, 212, 213, ..., 214, respectively, correspond to one or more different zones 211, 212, 213, ..., 214 of the target surface 12. m This is represented by dashed and solid arrows 201, 202, 203, ..., 20 k As can be seen, the first motion class 151 may be associated with a first zone 211 of the object surface 12, as indicated by the dashed arrow 201. The second motion class 152 may be associated with a second zone 212 of the object surface 12, as indicated by the dashed arrow 202. The third motion class 153 may be associated with a third zone 213 of the object surface 12, as indicated by the solid drawn arrow 203. k is dashed arrow 20 k As shown by the mth zone 21 of the target surface 12, mThis is generally known to those skilled in the art of neural networks, but here the neural network is first trained with labeled input movement patterns, i.e., movement patterns known to be associated with a particular movement class, so that the neural network used in the device 10 will of course predict with a certain prediction probability the movement classes 151, 152, 153, ..., 154, which are included in the set of (trained) movement classes 15. k It is described that a trained neural network can classify new input movement patterns for a given object.
[0030] The solid arrow 203 indicates that the neural network 18 is connected to at least one of the inertial sensor data 171, 172, 173, ..., 17 n , 152, 153, ..., 154 of the object surface 12. n and zones 211, 212, 213, …, 21 m The differentiation made here between the motion classes 151, 152, 153, ..., 154, ..., 155, ..., 156, ..., 157, ..., 158, ..., 159, ..., 160, ..., 161, ..., 162, ..., 163, ..., 164, ..., 165, ..., 166, ..., 167, ..., 168, ..., 169, ..., 170, ..., 171, ..., 172, ..., 173, ..., 174, ..., 175, ..., 176, ..., 177, ..., 178, ..., 179, ..., 180, ..., 181, ..., 182, ..., 183, ..., 184, ..., 185, ..., 186, ... k For a first number of zones, say k=20, it is trained to classify the input (i.e., movement patterns) and the output of the neural network is trained to classify the inputs of the zones 211, 212, 213, ..., 214, ... m, where m=6. As will be explained further below, one or some of the motion classes can be associated with more than one zone (e.g., motion class 5 can be associated with zones 3 and 4). It should be clear from the above that the NN does not classify the input motion patterns for zones, but only for motion classes, and it is reiterated here that the step of identifying zones may involve various additional computational steps (explained in more detail below), in particular that the number of zones m may be lower (or higher, which means different) than the number of motion classes k. More specifically, the device can output a zone for each input vector sequence, for example, the device can output zone 6 as the zone in which the device was used during the period associated with the current input vector sequence. As will be explained further below, a maximum or large criterion can be used to map the output vector sequence to a single zone output.
[0031] Therefore, at least one motion class 151, 152, 153, …, 15 k At least one inertial sensor data 171, 172, 173, ..., 17 n The classification is performed by dividing the target surface 12 into one or more zones 211, 212, 213, ..., 21 m In this example, the estimation of the position of the mobile treatment device 11 relative to the third motion class 153 is performed using at least one inertial sensor data 171, 172, 173, ..., 174. n The classification of y represents an estimate of the position of the mobile treatment device 11 relative to the third zone 213 of the target surface 12. As will be explained in more detail below, the output vector y t typically provides a probability value that the input motion pattern is associated with each of the motion classes.
[0032] In other words, the neural network 18 receives at least one inertial sensor data 171, 172, 173, ..., 17 n was successfully classified into the third motion class 153 (the output vector y t According to this example, the third motion class 153 is associated with the third zone 213, so the apparatus 10 determines that the mobile treatment device 11 may be located in the third zone 213 or that at least one of the inertial sensor data 171, 172, 173, ..., 174 may be located in the third zone 213. n The mobile treatment device 11 is located in at least the third zone 213 at the time the map was created.
[0033] Thus, the apparatus 10 may be configured to locate the mobile treatment device 11 relative to the target surface 12 simply by means of an executed movement or movement pattern of the mobile treatment device 11 .
[0034] According to one embodiment, the mobile treatment device 11 may be a personal instrument and the target surface 12 may be a body part to be treated by the mobile treatment device 11 .
[0035] For example, the mobile treatment device 11 is a razor or groomer for shaving or grooming body parts of a user's body. In this case, the user's body may be the target surface 12. The user's body 12 may be separated into different zones, for example, a left cheek zone, a right cheek zone, a chin zone, etc. By performing a specific movement pattern with the razor 11, the apparatus 10 can localize the razor 11 relative to the user's body using a trained neural network.
[0036] As a further example, the mobile treatment device 11 may be a household appliance and the target surface 12 may be a surface such as a floor, a wall, a piece of furniture, etc. For example, the mobile treatment device 11 may be a vacuum cleaner and the target surface 12 may be a floor of a room. The room 12 may be separated into different zones, for example, the upper left corner of the room, the lower right corner of the room, the center of the room, under a bed located inside the room, etc. By executing a specific motion pattern with the vacuum cleaner 11, the apparatus 10 may localize the vacuum cleaner 11 relative to the floor of the room. For example, if the vacuum cleaner 11 executes a motion pattern directed simply forward and backward with the lance of the vacuum cleaner 11 lowered close to the ground, the apparatus 10 may localize the vacuum cleaner 11 to be located, for example, in the "under the bed" zone. Thus, the apparatus 10 may easily localize the vacuum cleaner 11 inside the room by its executed motion pattern.
[0037] According to a further embodiment, the mobile treatment device 11 is an oral care device and the target surface 12 may be an oral cavity, the oral cavity 12 being divided into different zones 211, 212, 213, ..., 214. m and at least one motion class 151, 152, 153, …, 15 k At least one inertial sensor data 171, 172, 173, ..., 17 n The classification of the oral cavity 12 includes one or more oral cavity zones 211, 212, 213, ..., 21 m 1 shows an estimation of the position of the oral care device 11 relative to the dentition. The oral cavity notably comprises the dentition, which may be separated into different dental zones (as will be explained in further more detail below), but also the tongue (in particular the tongue may be separated into different tongue sections), and the gingival and cheek sections, whose areas may be zones as well, and movement classes may be trained such that the neural network can classify input movement patterns into movement classes associated with the tongue, etc.
[0038] The oral care device may be a toothbrush, in particular a power toothbrush. The oral care device may also be at least one of a dental floss, a plaque removal device, an ultrasonic device, and a water jet device.
[0039] According to this example, by performing a specific motion pattern with the oral care device 11, the apparatus 10 can locate the oral care device 11 relative to the oral cavity, for example, relative to the dentition. For example, if the oral care device 11 performs a motion pattern directed simply upwards and downwards, with the oral care device 11 tilted to the left, the apparatus 10 can locate the oral care device 11 to be located in the upper left dental zone of the upper jaw, for example, by classifying the motion into learned motion classes and further mapping the identified motion classes onto the zones of either each time instant or the entire time sequence of the input data. Thus, the apparatus 10 can easily locate the oral care device 11 relative to the user's dentition by its performed motion pattern. The input motion patterns described herein as examples are intended to illustrate only basic concepts. In fact, the motion patterns associated with a particular zone (e.g., upper front teeth) depend heavily on the individual and the individual's brushing habits. For example, the individual may be a right-handed person or a left-handed person, and the individual may be scrubbing. It is clear that the training phase of the neural network can include so many different individual or user types and situations that affect brushing habits that the predictive quality of the neural network can be high for all types of input movement patterns.
[0040] According to one embodiment, the dentition can be separated into nine dental zones, a first dental zone corresponding to the left buccal side of the maxilla and mandible of the dentition, a second dental zone corresponding to the left and right occlusal side of the maxilla of the dentition, a third zone corresponding to the left and right occlusal side of the mandible of the dentition, a fourth dental zone corresponding to the left lingual side of the maxilla and mandible of the dentition, a fifth dental zone corresponding to the right buccal side of the maxilla and mandible of the dentition, a sixth dental zone corresponding to the right lingual side of the maxilla and mandible of the dentition, a seventh dental zone corresponding to the labial side of the maxilla and mandible of the dentition, an eighth dental zone corresponding to the palatal side of the maxilla of the dentition, and a ninth dental zone corresponding to the oral side of the anterior mandible of the dentition.
[0041] According to a further embodiment, at least one exercise class 15 may additionally be included in the set of exercise classes 15. NB is the outer zone 21 of the target surface 12 NB 1, and may be associated with at least one motion class 15, which may be associated with the target surface 12 or may not be associated with the target surface 12. NB At least one inertial sensor data 171, 172, 173, ..., 17 n The mapping of the above zone 21 may be performed when the mobile treatment device 11 is outside the target surface 12 or is not associated with the target surface 12. NB Indicates that the device is located in the 15th physical activity class. NB may be associated with an input motion pattern in which a user moves the movable treatment device 11 toward a target surface (e.g., the oral care device is placed on a bathroom sink and moved toward the oral cavity) or in which a user transitions the movable treatment device 11 from one zone to another (e.g., the user can move the oral care device from the lower left molar toward the lower right molar).
[0042] In other words, the outer zone 21 of the target surface 12 NB may be a zone not directly related to the target surface 12. For example, if the mobile treatment device 11 can be a toothbrush, the above-mentioned zone 21 outside the target surface 12 NB can be the outer zone of the dentition. Therefore, this zone 21 NBmay indicate that the user does not brush his / her teeth. Therefore, this zone is sometimes referred to as the "no brushing" zone, abbreviated as "NB". This zone 21 NB may be at least one zone of the target surface 12, or this zone 21 NB , 211, 212, 213, ..., 21 m However, this particular zone 21 outside the target surface 12 may be an additional zone. NB is not limited to the exemplary tooth brushing described above. It should not be excluded that the motion pattern classification device includes a set of motion classes, and one or more of the motion classes are each associated with one or more zones outside the target surface, for example, at least one motion class can be associated with the outer zone of the target surface (for example, when the device is moved toward or away from the target surface), and / or at least one motion class can be associated with the transition between zones of the target surface (for example, when the device is moved from zone 3 to zone 4).
[0043] FIG. 2 shows a dentition 12 to illustrate the examples described above. The dentition 12 may be a target surface or may be a part of a target surface, which may also include, for example, the tongue. The dentition 12 may be separated into nine dental zones 1a-9a. Optionally, a tenth zone NB may be present. This tenth zone NB is an outer zone of the dentition 12. This tenth zone NB is therefore not explicitly illustrated in FIG. 2. As this tenth zone NB does not relate to one of the dental zones of the dentition 12 and thus does not relate to the brushing of the teeth of the dentition 12, this tenth zone NB may also be referred to as a "non-brushing" zone.
[0044] As can be seen in FIG. 2, the first dental zone 1a may correspond to the left buccal side of the upper and lower jaws of the dentition 12. The second dental zone 2a may correspond to the left and right occlusal side of the upper jaw of the dentition 12. The third zone 3a may correspond to the left and right occlusal side of the lower jaw of the dentition 12. The fourth dental zone 4a may correspond to the left lingual side of the upper and lower jaws of the dentition 12. The fifth dental zone 5a may correspond to the right buccal side of the upper and lower jaws of the dentition 12. The sixth dental zone 6a may correspond to the right lingual side of the upper and lower jaws of the dentition 12. The seventh dental zone 7a may correspond to the labial side of the upper and lower jaws of the dentition 12. The eighth dental zone 8a may correspond to the palatal side of the upper jaw of the dentition 12. The ninth dental zone 9a may correspond to the oral side of the anterior lower jaw of the dentition 12. A further zone may correspond to the tongue.
[0045] FIG. 3 shows a dentition 12 to illustrate a further example. The dentition 12 may be a target surface. The dentition 12 may be separated into 16 dental zones 1b-16b. Optionally, a 17th zone NB may be present. This 17th zone NB is an outer zone of the dentition 12. This 17th zone NB is therefore not explicitly illustrated in FIG. 3. As this 17th zone NB does not relate to one of the dental zones of the dentition 12 and thus does not relate to the brushing of the teeth of the dentition 12, this 17th zone NB may also be referred to as a "no brushing" zone.
[0046] As can be seen in FIG. 3, the first dental zone 1b may correspond to the left buccal side of the maxilla of the dentition 12. The second dental zone 2b may correspond to the left occlusal side of the maxilla of the dentition 12. The third dental zone 3b may correspond to the left occlusal side of the mandible of the dentition 12. The fourth dental zone 4b may correspond to the left lingual side of the maxilla and mandible of the dentition 12. The fifth dental zone 5b may correspond to the right buccal side of the maxilla and mandible of the dentition 12. The sixth dental zone 6b may correspond to the right occlusal side of the maxilla of the dentition 12. The seventh dental zone 7b may correspond to the right occlusal side of the mandible of the dentition 12. The eighth dental zone 8b may correspond to the palatal side of the maxilla of the dentition 12. The ninth dental zone 9b may correspond to the labial side of the maxilla of the dentition 12. The tenth dental zone 10b may correspond to the labial side of the mandible of the dentition 12. The eleventh dental zone 11b may correspond to the palatal side of the upper jaw of the dentition 12. The twelfth dental zone 12b may correspond to the oral side of the anterior mandible of the dentition 12. The thirteenth dental zone 13b may correspond to the left buccal side of the mandible of the dentition 12. The fourteenth dental zone 14b may correspond to the left lingual side of the mandible of the dentition 12. The fifteenth dental zone 15b may correspond to the right buccal side of the mandible of the dentition 12. The sixteenth dental zone 16b may correspond to the right lingual side of the mandible of the dentition 12.
[0047] In this example, 16 motion classes can be trained (or 17 motion classes including a no-brushing motion class), and a bijective function can be used to associate the motion classes and the zones with each other. Referring to the oral care example, further motion classes can be trained. For example, it is known that a user tends to brush the outside of the left molar (i.e., the buccal surface of the left molar) together, instead of brushing the buccal surface of the upper left molar and the buccal surface of the lower left molar separately. Thus, the neural network can be trained to classify the input motion pattern to a further motion class associated with brushing this combination of the buccal surface of the left molar. This motion class is then associated with two of the aforementioned zones, namely the first dental zone 1b and the thirteenth dental zone 13b. Thus, the further trained motion class can be associated with the outside surfaces of the anterior teeth and the buccal surface of the right molar. In such an example, 20 motion classes are used and associated with 17 zones.
[0048] 2 and 3 are described only as non-limiting examples. The target surface 12 may also comprise more or less dental zones than the 9 or 16 dental zones described. Furthermore, the outer 10th / 17th dental zones NB of the target surface 12 are optional. The exact distribution of one or more dental zones of the dentition 12 may differ from the examples described above.
[0049] The neural network 18 may be a recurrent neural network (RNN). For example, the neural network may be a Long Short-Term Memory (LSTM) network or a Gated Recurrent Unit (GRU) network. The RNN may be a bidirectional recurrent neural network. This means that a sequence of input vectors is fed to a first recurrent neural network that computes the output from left to right (e.g., past to future) and a separate second recurrent neural network that computes the output from right to left (e.g., future to past). In this way, the RNN can use past and future features in the input sequence. The forward and backward hidden vectors of the two separate RNNs are then fed together to an output layer to generate an output vector sequence.
[0050] Alternatively, the device 10 can use two different neural networks. With reference to the example of a dentition, a first neural network can classify the input movement pattern into the movement classes of the left molar, right molar, and anterior teeth, and a second neural network can classify the input movement pattern into the movement classes of the upper and lower jaws. This segmentation may be called an orthogonal movement class. The outputs of the first and second neural networks can then be combined to output one of the six zones of the dentition, namely upper left molar, upper right molar, lower left molar, lower right molar, upper anterior teeth, or lower anterior teeth. This represents an alternative to directly using a single neural network that outputs probability values of at least the six movement classes mentioned. The use of two neural networks may be extended by further neural networks, for example a third neural network can classify the input movement pattern into the movement classes of the buccal and lingual (and occlusal) surfaces of the teeth.
[0051] As previously mentioned, an interface can be arranged to provide a sequence of inertial sensor data to the motion pattern classification device, where the sequence is related to a time sequence. For tooth brushing, it has been found that a sequence length in the range of 20-30 inertial sensor data represents a sensible sequence length, and the period over which these inertial sensor data are sampled can be in the range of 0.5 seconds to 5 seconds, and in particular can be about 1 second.
[0052] RNNs can suffer from the so-called vanishing gradient problem, where the gradient vanishes quickly with a larger number of layers. Vanishing gradients can result in slower training speed. Therefore, LSTM networks and / or GRU networks can be used to avoid the vanishing gradient problem.
[0053] An LSTM network is an artificial neural network that includes an LSTM module or unit in addition to regular network units (such as a fully connected output layer). Here, the terms LSTM module, or LSTM unit, or LSTM cell, or LSTM block are all intended to mean the same as the exemplary depicted in FIG. 6a, i.e., a complete LSTM unit. The LSTM module includes a gate that determines when the input is large enough to cause the LSTM unit to remember a value, when to keep remembering the value, or when to forget the value.
[0054] 4 shows an example of a simple form of RNN module 40. Module 40 (sometimes also called a neural unit) can be provided with an input 41 at a particular time instant t. Input 41 can be a single value or a vector containing two or more values. The input 41 at a particular time instant t can be denoted as X t The value(s) of input 41 may be raw data from an inertial sensor. If a 6-axis inertial sensor (e.g., a 3-axis accelerometer and a 3-axis gyroscope) is used, input 41 may have 6 values, i.e., vector Xt is the six vector elements, or in other words, X t ∈R 6 has.
[0055] As already mentioned, the mobile treatment device may comprise further sensors, for example a pressure sensor measuring the pressure with which the mobile treatment device is pressed against the treatment surface. In such an example, the pressure can be sampled in the same way, in which case the input 41 may for example include a seventh element, i.e. a pressure value, which, if a single pressure value is assumed, is X t ∈R 7 The pressure (or force) sensor may instead be a multi-axis pressure (or force) sensor, in which case the input may have even higher dimensions. The pressure values may improve the classification quality of the neural network. The neural unit 40 may also include a further input 42. This further input 42 may be provided from a neural unit (not depicted here) at a previous time instant t-1. The calculation direction of the recurrent neural network does not have to be operated in the positive time direction (i.e. from the past to the future). If a sequence of inertial sensor data is input to the RNN, the recurrent neural network may calculate from the last inertial sensor data in time towards the first inertial sensor data in time (i.e. from the future to the past) or vice versa. As already mentioned, the RNN may be a bidirectional RNN with one branch that is calculated from the past to the future and another branch that is calculated from the future to the past. This means that the term "previous time instant" refers only to a previous time instant with respect to the calculation direction, but not necessarily to an earlier time instant.
[0056] The neural unit 40 may include at least one neural network layer 43 that may provide mathematical operations. In this example, the gate 43 is a single tanh layer. It should be understood that the neural network layers shown as boxes indicate trained network layers.
[0057] The neural unit 40 may include at least one output 46. The output 46 may include the result of the operation of a tanh neural network layer 43 supplied with the inputs 41 and, optionally, further inputs 42. The output 46 may result in a hidden state 45, which will be described later.
[0058] The neural unit 40 may optionally include a further output branch 46 branching off from the above-mentioned output result of the operation of the tanh gate 43 supplied with the input 41 and optionally a further input 42 .
[0059] In Figure 4, each drawn line may carry an entire vector from the output of one node to the input of another. The joining of lines, e.g. at 47, indicates a concatenation, whereas the forking of lines, e.g. at 48, indicates that the contents are copied and the copies go to different locations. This is also true for the other neural network units described below with reference to the following figures.
[0060] This is commonly known to those skilled in the art of neural networks, but it is useful to consider an input vector X that can contain six values. t can be mapped onto vectors with much higher dimensions in the input layer of the module 40, for example the vectors used in the NN module 40 are 256 It should be understood here that this is merely a non-limiting example. Typical values for the dimension of vectors in neural networks might be 128, 256, 512, or 1024, although other values could be chosen as well. Thus, the input weighting matrix for implementing this mapping is R 6 xR 256 A bias vector may be used as known in the art, in which case this bias vector is also R 256 The output vector h of the module 40 can be t Also, R 256Assuming that the RNN has only a single repeated chain of modules, as shown in Figure 4, then the output layer is the vector of output vector h t Each of these is expressed as the number of motion classes, e.g., R 20 , the output weighting matrix can then be mapped onto an output vector with dimensions R 256 xR 20 As mentioned before, the weights and biases are determined in the training phase.
[0061] 5 shows an example of a GRU module 50 that may be used in an RNN. In addition to the simple RNN neural unit 40 described above, the GRU neural unit 50 may comprise two further neural network layers: a first sigmoid layer 53 and a second sigmoid layer 54. Additionally, the GRU neural unit 50 may include pointwise operations 55, 56, 57, 58, 59, such as vector addition 58.
[0062] 6a shows an example of a recurrent module 60, commonly known as an LSTM module, that may be utilized in the neural network 18 in the apparatus 10 according to the present invention. In addition to the neural units 40, 50 described above, the LSTM unit 60 may include a cell state, which is a horizontal line 61 running through the top of the neural unit 60. The neural unit 60 may receive a cell state input 62 and may produce a cell state output 66.
[0063] The neural unit 60 may comprise three gates (or gate layers): an input gate 54, an output gate 63, and a forget gate 53. Each of these gates 53, 54, 63 can be thought of as a "standard" neuron in a feedforward (or multi-layer) neural network. That is, they calculate a weighted sum activation (using an activation function). Information can be removed or added by these gates 53, 54, 63 up to the cell state (horizon 61). The input gate 54 is combined with a tanh layer 43, which creates candidate states that together with the input gates determine the update of the cell state. See also the set of equations further shown and discussed below that describe the flow within the LSTM unit.
[0064] It should be understood that neural units 50 and 60 are merely shown and discussed as exemplary neural units. Other neural units may use peephole connections and / or connected forgetting and input gate layers. Long-term dependencies may also be approached by, for example, a Clockwork RNN or other multi-scale RNN.
[0065] FIG. 6b shows an example of an unrolled RNN with an LSTM module in which the previous and subsequent states (with respect to time instant t) of a neural unit are depicted. Specifically, the neural unit 60 at time instant t t Further, a further neural unit 60 at the previous time instant t-1 is depicted. t-1 Further, at a subsequent time instant t+1, a further neural unit 60 t+1 The neural units depicted are 60. t-1 , 60 t , 60 t+1 can represent the same neural unit but at different points in time, i.e., at time instant t, a previous time instant t−1 and a subsequent time instant t+1. tIt can be understood as a portion of an unrolled LSTM network with as many recurrent neural units as there are in the case of . As mentioned before, sequence lengths in the range of 20-30, e.g., about 25, can be used.
[0066] The input 41 described above may also be symbolized by the letter X, and includes at least one sensor data 171, 172, 173, . . . , 174 from the inertial sensor 13. n The input X may be time-dependent, so that X=X(t). Specifically, the input X to be depicted t may include the sensor data 172 acquired during the considered time instant t, and the depicted input X t-1 may include sensor data 171 acquired during a previous time instant t−1, and the depicted input X t+1 may include sensor data 173 acquired during the subsequent time instant t+1. As explained above, the input X is fed to the input layer of the RNN.
[0067] As can be further seen in FIG. 6b, neural unit 60 t-1 , 60 t , 60 t+1 are the output values y at each depicted time instant t-1, t, and t+1. t-1 , y t , y t+1 The output value y(t) may be a single value (if only one single motion class exists) or a vector containing multiple vector elements.
[0068] The output value y(t) can be calculated in the fully connected output layer of the RNN as follows: y t = Softmax(W hy h t +b) In the formula, W hywhere x is a respective output weighting matrix and b is a respective output bias vector. The output vector y(t) includes probability values ranging from 0 to 1, as will be described in more detail with respect to FIG. 7. The output vector y(t) includes one or more vector elements, each of which is a function of the input vector X t , i.e., inertial sensor data 171, 172, 173, ..., 17 n Exercise classes 151, 152, 153, …, 15 k It is to be understood that the application of the softmax function is merely optional. In particular, the identification of the motion class with the highest probability in the output vector can be done without any normalization.
[0069] Further, the depicted neural unit 60 t-1 , 60 t , 60 t+1 may be located in the same layer, i.e., the first layer. Some examples of the present invention may include one or more further stacked layers, each layer may include its own neural unit(s). Such examples may be described below, for example, with reference to FIG. 7. However, examples and embodiments having at least a first layer will be described with further reference to FIG. 6b.
[0070] According to this embodiment, the neural network 18 includes a neural unit 60, here an LSTM unit. t-1 , 60 t , 60 t+1 The first layer may include a first layer of neural units 60 t At a first time instant t, at least one inertial sensor data X t The first layer of neural units is 60 t At a subsequent second time instant t+1, the second inertial sensor data X t+1 , and the neural unit 60 at the first previous time instant t t At least one output of h t The first layer of neural units is 60 t+1is entered into
[0071] 7 shows a further example in which the neural network 18 comprises at least two stacked layers of neural units: a first layer 71 and a second layer 72. The first layer 71 comprises at least a first neural unit 60. t The second layer 72 includes at least a second neural unit 70 t The stacked structure shown can be understood to be a deep RNN structure comprising multiple stacked RNN hidden layers.
[0072] As can be seen, the sensor data 171, 172, 173 acquired at different time instances t−1, t, t+1 are t-1 , X t , X t+1 As a result, each neural unit 60 in the first layer 71 t-1 , 60 t , 60 t+1 are supplied to.
[0073] Each neural unit 60 in the first layer 71 t-1 , 60 t , 60 t+1 Hidden output vector 45 t-1 , 45 t , 45 t+1 is input to each neural unit 70 of the second layer 72. t-1 , 70 t , 70 t+1 can be supplied to
[0074] Neural units 60 of the first layer 71 t-1 , 60 t , 60 t+1 and the neural units 70 of the second layer 72 t-1 , 70 t , 70 t+1 Alternatively, the neural units 60 of the first layer 71 may be the same. t-1 , 60 t , 60 t+1 and the neural units 70 of the second layer 72 t-1, 70 t , 70 t+1 The internal structures of the may differ from each other.
[0075] According to the embodiment shown in FIG. 7, the neural network 18 may comprise at least a first layer 71 and a second layer 72, the first layer 71 including a first neural unit 60 t The second layer 72 may include a second neural unit 70 t At a first time instant t, at least one inertial sensor data X t The first neural unit 60 of the first layer 71 t is input to the first neural unit 60 t Output of h t However, the neural unit 70 of the second layer 72 t In the case of a bidirectional RNN, the intermediate hidden output vectors 45 t can be fed to the neural units of the next layer.
[0076] So far, signal paths have been described vertically, i.e., from the first layer 71 at the bottom to the second layer 72 at the top. However, in the embodiment of Figure 7, horizontal signal paths are also shown.
[0077] As can be seen, the first neural unit 60 at the first time instant t t Cell status output C t , and / or the first neural unit 60 at the first time instant t t Output of h t 46 is again provided as an input to the first neural unit 60, i.e., at a subsequent time instant t+1, t+1 As already mentioned above, the neural unit 60 itself may be the same neural unit, but in order to facilitate illustration of the state of the neural unit 60 at different time instances t−1, t, t+1, the neural unit 60 may be t-1 , 60 t , 60t+1 t−1, t, t+1. In other words, the horizontal signal paths may describe the signal paths of neural unit 60 at different subsequent time instances t−1, t, t+1. The same is true for second layer 72 and any potential further layers.
[0078] Thus, the subsequent time instances t−1, t, t+1 depicted are the time instances t−1, t, t+1 depicted as the neural network 18 iterates over the acquired sensor data 171, 172, 173, ..., 17 n may represent a length 77 during which neural network 18 may be sampled and processed. The length 77 may therefore be referred to as a run length, a sample length, or a sample period. For example, sample length 77 may correspond to one second, and time instances t-1, t, t+1 may be fractions of a second. For example, sample period 77 may be 50 samples long, i.e., have 50 time instances. Neural network 18 may be operated once during a sample period, or neural network 18 may be operated permanently for more than one sample period.
[0079] Thus, according to a further embodiment, the neural network 18 may comprise at least a first layer 71 and a stacked second layer 72, the first layer 71 including a first neural unit 60 t The second layer 72 may include a second neural unit 70 t At a first time instant t, at least one inertial sensor data X t The first neural unit 60 of the first layer 71 t , and the first neural unit 60 t At least one output of h t However, the neural unit 70 of the second layer 72 t Thus far, it may be the same as described above. However, in addition, the second inertial sensor data X at a subsequent second time instant t+1 may be input to t+1, and a first neural unit 60 at a first time instant t t At least one output of h t 46 is a first neural unit 60 at a subsequent second time instant t+1. t+1 is entered into
[0080] As mentioned above, several mathematical operations can be performed by the neural network 18, for example in gates 43, 53, 54, 63. In the example shown in FIG. 7, the following mathematical operations can be performed in different stages:
[0081]
number
[0082] Additionally, weighting matrices W, U and bias vectors b are used at each network layer, in this example: Input weighting W xy ∈R 6×256 , Hidden layer weights U xy ∈R 256×256 , Hidden layer b x ∈R 256 is the bias used in
[0083] Hidden state h t also has 256 elements h t ∈R 256 is an element vector containing
[0084] Furthermore, cell state c t also has 256 elements c t ∈R 256 is an element vector containing
[0085] As mentioned above, the input inertial sensor data X t returns 6 vector elements, e.g., the input tensor X t ∈R 6 , [A x , A y , A z , G x , G y , G z ] T Element vector X containing t ∈R 6 These vector elements [A x , A y , A z , G x , G y , G z ] T is also sometimes referred to as the inertial sensor data portion.
[0086] According to one embodiment, the at least one inertial sensor data 171 may include at least three inertial sensor data portions of the group including linear velocity in the x, y, and z directions, angular velocity about the x, y, and z axes, linear acceleration in the x, y, and z directions, and angular acceleration about the x, y, and z axes.
[0087] In other words, the inertial sensor 13 generates inertial sensor data 171, 172, 173, ..., 174 at one or more time instances t-1, t, t+1. n , and inertial sensor data 171, 172, 173, ..., 17 n may depend on the current orientation and motion of the mobile processing device 11 at one observed time instance t-1, t, t+1. n may each be a vector including at least three, or in other examples, at least six, vector elements, the vector elements representing the inertial sensor data portions described above, and at least one of the inertial sensor data portions may be zero.
[0088] Therefore, the inertial sensor data 171, 172, 173, ..., 17 n (vector), and in particular the sensor data portions (vector elements), may represent the current movement pattern of the mobile treatment device 11, sampled during a sample period 77 that includes one or more subsequent time instances t-1, t, t+1.
[0089] According to one embodiment depicted in FIG. 7, at least one inertial sensor data 172 (vector) may include one or more inertial sensor data portions (vector elements) and may be input to the neural unit 60 at a first time instant t. t The input to the first time instant t is a respective inertial sensor data 172, which includes one or more inertial sensor data portions collected during the first time instant t. n may be sampled during sample time 77.
[0090] The neural network 18 is configured to generate a set of motion classes 151, 152, 153, ..., 154, as first described with reference to FIG. k At least one sampled inertial sensor data 171, 172, 173, ..., 17 sampled during sample time 77 n can be classified as follows.
[0091] As stated, the output vector y of the neural network t may contain vector elements each having a value in the range 0 to 1, which sum up to 1 when the optional softmax function is applied. Thus, each vector element is t Exercise classes 151, 152, 153, …, 15 k , . . . , . . . The motion pattern classification device 14 may indicate the motion class for which the input received the highest probability value as the associated motion class. In some embodiments, the motion pattern classification device 14 may not indicate a motion class if the highest probability value is below a certain threshold, for example, below 0.7 (for normalized output). Values below the threshold may be seen to indicate low prediction quality and unreliable results. In some embodiments, the motion pattern classification device 14 may indicate the motion class for the complete sequence of input vectors, and the indicated motion class was identified by applying a maximum or large criterion on the sequence of output vectors.
[0092] In other words, the neural network 18 uses the inertial sensor data 171, 172, 173, ..., 17 n may be received as an input x(t) and one or more probability values may be output as an output y(t). As mentioned above, in the example shown in Figure 7, the output value y(t) may also be an element vector, for example including at least 3, or at least 6, or at least 12 vector elements. Each vector element of the output vector y(t) is a vector element of a zone 211, 212, 213, ..., 214.m Exercise classes 151, 152, 153, ..., 15 k This means that when the movement pattern classification device 14 provides as output a single movement class, e.g. by selecting the movement class with the highest probability value above a threshold, the associated zone or zones in which the mobile treatment device was used during the sampling period may be indicated to the user, e.g. on a display.
[0093] The neural network 18 uses at least one inertial sensor data 171, 172, 173, ..., 17 n Receive at least one exercise class 151, 152, 153, ..., 15 k At least one inertial sensor data 171, 172, 173, ..., 17 n The above motion classes 151, 152, 153, ..., 15 k , 211, 212, 213, ..., 214, respectively, correspond to one or more different zones 211, 212, 213, ..., 214 of the target surface 12. m , 171, 172, 173, . . . , 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 1 n , 211, 212, 213, ..., 21 m It is possible to show the probability that
[0094] In other words, the apparatus 10 simply receives the sensor data 171, 172, 173, ..., 17 n , each of which corresponds to one or more zones 211, 212, 213, ..., 21 m Associated exercise classes 151, 152, 153, …, 15 k For the above sensor data 171, 172, 173, ..., 17 n By classifying the zones 211, 212, 213, ..., 21 m An estimate can be derived where the mobile treatment device 11 is located.
[0095] FIG. 8 shows a block diagram of an example of a method of the present invention for performing localization of a mobile treatment device 11 relative to a target surface 12, the mobile treatment device 11 being equipped with an inertial sensor 13 and configured to treat the target surface 12.
[0096] In block 801, the method includes: k Here, discrimination is intended to mean that the motion pattern classifier device 14 is trained to classify the input motion pattern into two or more motion classes. In other words, two or more motion classes are provided, from which classification can be performed.
[0097] In block 802, the method includes receiving at least one inertial sensor data 171, 172, 173, . . . , 177 from the inertial sensor 13. n receiving at least one inertial sensor data 171, 172, 173, ..., 17 n represents the movement of the mobile treatment device 11.
[0098] In block 803, the method includes computing, by the neural network 18, at least one inertial sensor data 171, 172, 173, ..., 17 n , 152, 153, . . . , 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 1 k At least one inertial sensor data 171, 172, 173, ..., 17 n , 151, 152, 153, ..., 155 included in a set of motion classes 15. k , 211, 212, 213, ..., 214, respectively, correspond to one or more different zones 211, 212, 213, ..., 214 of the target surface 12. m , and thereby at least one motion class 151, 152, 153, ..., 15 k At least one inertial sensor data 171, 172, 173, ..., 17n The classification is performed by dividing the target surface 12 into one or more zones 211, 212, 213, ..., 21 m A method showing an estimation of a position of a mobile treatment device 11 relative to the
[0099] 9 shows an apparatus 100 according to a second embodiment of the present invention. The apparatus 100 of the second embodiment may be similar to the apparatus 10 of the first embodiment described above. Furthermore, all of the features described above with respect to the apparatus 10 of the first embodiment may be combined with the apparatus 100 of the second embodiment described below, and vice versa.
[0100] The device 100 of the second embodiment is divided into exercise classes 151, 152, 153, ..., 15 k , 211, 212, 213, ..., 21 m , mB。 In essence, this is a more abstract and broader consideration of the basic concepts discussed with respect to the device 10 as the associated class 101 may be appropriate. In the following description, the focus is on associated classes 101 that are related to users, and in particular associated classes 101 that may be related to different members of a group of users (e.g., different members of a household) or users that have different styles of using the mobile treatment device 11.
[0101] Thus, the apparatus 100 of the second embodiment is configured to classify the motion of a mobile treatment device 11 equipped with an inertial sensor 13. The apparatus 100 classifies two or more motion classes 151, 152, 153, ..., 154 of the set of motion classes 15 of the mobile treatment device 11. k The device further includes a movement pattern classification device 14 configured to distinguish between
[0102] Furthermore, the device 100 receives at least one inertial sensor data 171, 172, 173, ..., 17 from the inertial sensor 13.n , 171, 172, 173, . . . , 174, to the motion pattern classification device 14. n represents the movement of the mobile treatment device 11.
[0103] According to a second aspect of the present invention, the movement pattern classification device 14 classifies the movement pattern by using at least one inertial sensor data 171, 172, 173, ..., 17 n , 151, 152, 153, . . . , 155, and at least one of the motion classes 151, 152, 153, . . . , 155 included in the set of motion classes 15. k At least one inertial sensor data 171, 172, 173, ..., 17 n , 151, 152, 153, ..., 15 k is associated with at least one or more class members 101A, 101B, 102A, 102B, 103A, 103B, mA, mB of one or more associated classes 101, 102, 103, 104, whereby at least one class member 101A, 101B, ..., mA, mB is selected based on the movement of the mobile treatment device 11.
[0104] Classification of the input motion pattern, i.e., the motion classes 151, 152, 153, ..., 15 included in the set of motion classes 15 k Inertial sensor data 171, 172, 173, ..., 17 n Regarding this, reference is made to the previous explanation.
[0105] 9 shows four associated classes 101, 102, 103, 104, each associated class including two class members 101A, 101B, ..., mA, mB. However, there may be at least one associated class, and each associated class may include at least two class members. Also, there may be more than two associated classes, or even more than the four exemplary depicted associated classes.
[0106] As can be seen in the example of FIG. 9, the first motion class 151 may be associated with a class member 101A of the first associated class 101. k may be associated with class member mB of the mth class 104. The second motion class 152 may be associated with two class members of different associated classes, e.g., class member 101B of the first associated class 101 and class member 102A of the second associated class 102. The third motion class 153 may be associated with two class members of the same associated class, e.g., two class members 103A, 103B of the third associated class.
[0107] Below are some examples of associated classes and class members.
[0108] According to one embodiment, at least one associated class 101 of one or more associated classes 101, 102, 103, 104 may include at least one class member 101A, the one associated class 101 may represent a user group, the at least one class member 101A may represent at least one user of the user group, and at least one exercise class 151, 152, 153, ..., 154 may represent at least one user of the user group. k may be associated with at least one class member 101A for identifying said at least one user based on the movement of the mobile treatment device 11.
[0109] In other words, one of the associated classes 101, 102, 103, 104 may represent a user group, i.e. a group of users who use the mobile treatment device 11. Each associated class may include at least one class member that may represent one particular user of said user group. For example, the first class 101 may represent a user group, said user group may relate to individuals of a single household. In this example, the user group 101 may include only one class member 101A, i.e. one person. The device 100 of the present invention may be configured to simply identify said at least one user 101A based on the movement of the mobile treatment device 11. Thus, the device 100 may personalize any action or interaction with said one identified user 101A, as will be explained later in some examples.
[0110] According to a further embodiment, at least one associated class 101 of the one or more associated classes 101, 102, 103, 104 may include at least two class members 101A, 101B, the one associated class 101 may represent a user group, the at least two class members 101A, 101B may represent at least two users of the user group, and at least one motion class 151, 152, 153, ..., 154 may represent at least two users of the user group. k may be associated with one of at least two class members 101A, 101B for identifying at least one user in a group of users based on the movement of the mobile treatment device 11.
[0111] In other words, one of the associated classes 101, 102, 103, 104 may represent a user group, i.e., a group of users who use the mobile treatment device 11. Each associated class may include at least one class member that may represent one particular user of the user group. For example, the first associated class 101 may represent a user group, which may be a family. The class members 101A, 101B of the associated class 101 may represent family members. For example, the user group 101 may include one or more family members, and the first class member 101A may represent the mother of the family, and the second class member 101B may represent the children of the family.
[0112] The apparatus 100 may be configured to simply identify at least one user based on the movement of the mobile treatment device 11. This may be achieved if all users are able to use the mobile treatment device 11 in different or individual ways.
[0113] For example, in one embodiment, the mobile treatment device 11 is a mobile oral care device such as a toothbrush, in particular a power toothbrush, etc. The mobile oral care device can also be at least one of a dental floss, a plaque removal device, an ultrasonic device, and a water jet device.
[0114] Taking the above example, mother 101A may use toothbrush 11 in a different way than child 101B. The inertial sensor 13 of toothbrush 11 may store its inertial sensor data 171, 172, 173, ..., 174, ... n can be provided to a movement pattern classification device 14 including a neural network 18. The neural network 18, as previously described, classifies at least one movement class 151, 152, 153, ..., 15 k For inertial sensor data 171, 172, 173, ..., 17 n can be classified as follows.
[0115] 9, a mother may have a brushing style that corresponds to a first exercise class 151. This exercise class 151 may be associated with a class member 101A that represents the mother. Alternatively, a child may have a different brushing style than the mother, for example, a brushing style that corresponds to a second exercise class 152. This exercise class 152 may be associated with a class member 101B that represents the child.
[0116] Therefore, the device 100 of the present invention can easily identify users of a group of users based on different movement patterns of the mobile treatment device 11. As mentioned above, the device 100 of the present invention can personalize any action or interaction with the identified users.
[0117] According to one embodiment, the movement pattern classification device 14, based on the step of identifying the at least one user 101A, classifies two or more user-specific movement classes 1151, 1152, 1153, ..., 1154 of the mobile treatment device 11 as characterized by the identified at least one user 101A. k , may be configured to select a user-specific set 115 including:
[0118] An example of this two-step process is shown in Figure 10. In the first step 121, users are identified. The identified users are assigned motion classes 1151, 1152, 1153, ..., 1154, which are individually trained by the neural network 18 using personalized labeled training data. k In a second step 122, the neural network 18 may associate user-specific exercise classes 1151, 1152, 1153, ..., 1154 from the user-specific set 115. k In other words, the neural network 18 may then use the weightings and biases associated with a particular user instead of the weightings and biases that are valid for a broader population. Thus, the device 100 of the present invention may act and interact individually for each identified user.
[0119] In FIG. 10, the neural network 18 computes the motion vectors based on at least one inertial sensor data 171, 172, 173, . . . 17 n , 152, 153, ..., 154, ..., 155, which are included in the set of motion classes 15. k A first step 121 is shown of classifying the k-th motion class into at least one of the k-th motion classes 15. k may be associated with the class member 101B of the first associated class 101. This procedure may correspond to the procedure described above with reference to FIG.
[0120] The associated class 101 may relate to a group of users and the class member 101B may be a particular user of said group of users. Taking the above example, the identified user 101B may be a child of a family. The device 100 may store user-specific exercise classes. That is, the identified user, i.e., the child 101B, may be associated with the exercise classes 1151, 1152, 1153, ..., 1154 stored in the device 100. k For any further actions following the identification in the first step 121, the movement pattern classification device 14, and in particular the neural network 18, may have its own individual user-specific set 115 of these user-specific movement classes 1151, 1152, 1153, ..., 1154 belonging to the previously identified user. k As mentioned previously, the reference to a motion class means that a user-specific set of weights and biases are used by neural network 18, where the weights and biases were trained using user-specific labeled input data.
[0121] Thus, after the step 121 of identifying the at least one user 101B, the neural network 18 may generate two or more user-specific movement classes 1151, 1152, 1153, ..., 1154 of the mobile treatment device 11 that are characterized by the identified at least one user 101B. kThe user may select at least one user-specific set 115 including:
[0122] Thus, following the first step 121 of identifying a user, in a second step 122, the neural network 18 identifies the motion classes 151, 152, 153, ..., 15 k Instead of the set 15 of user-specific exercise classes 1151, 1152, 1153, ..., 115 k A user-specific set 115 of exercise patterns 151, 152, 153, ..., 15 k All of the actions described herein may be performed by the device 10, 100 by utilizing a set 15 of movement patterns 151, 152, 153, ..., 15 k Instead of set 15, exercise classes 1151, 1152, 1153, …, 115 k The device 10, 100 may also implement an individualized or personalized approach for each identified user by utilizing a user-specific set 115 of the following:
[0123] Thus, according to one embodiment, the neural network 18, after the first step 121 of identifying the at least one user 101B, replaces the set of movement classes 15 with the selected user-specific set 115 and selects two or more movement classes 151, 152, 153, ..., 154 included in the set of movement patterns 15. k , 1151, 1152, 1153, ..., 1154, which are included in the preset 115 of user-specific exercise patterns. k The exercise classes 151, 152, 153, ..., 15 k were selected to predict users moving a mobile treatment device, while the motion classes 1151, 1152, 1153, ..., 115 k Note that may have been selected for a different task, such as locating the mobile treatment device 11.
[0124] Additionally or alternatively, the device 100 may include at least a second neural network. Figure 11 shows such an example.
[0125] The example of the apparatus 100 of Figure 11 may substantially correspond to the example apparatus 100 shown in Figure 10. The apparatus of Figure 11 differs from that of Figure 10 in that the apparatus of Figure 11 may comprise a second neural network 182. Although described with reference to Figure 10, it should be understood that a structurally identical neural network 18 is used for two different classification tasks, and that only the weights and biases may be changed, whereas the present example uses two different neural networks with different structures, thereby making it possible to use an optimized neural network structure for each classification task.
[0126] 11, in a first step 121, a first neural network 181 may perform the actions described above, for example, identifying a user 101B of a group of users 101. However, in a second step 122, the inertial sensor data 171, 172, 173, ..., 17 n may be fed to the second neural network 182. The second neural network 182 may be fed to the second neural network 182, which may be fed to the motion classes 1151, 1152, 1153, ..., 1154, as described above. n Obviously, a user-specific set of exercise classes 1151, 1152, 1153, ..., 115 n At least two different sets 115 of motion classes need to be stored in the device in order to run the second neural network with one set of motion classes optimized for one user and another set of motion classes optimized for another user.
[0127] In other words, after the first step 121 of identifying the at least one user 101B, the movement pattern classification device 14 can use a second neural network 182, which can be configured to classify the at least one inertial sensor data 171, 172, 173, ..., 174.n , 1151, 1152, 1153, . . . , 1155, and at least one user-specific exercise class 1151, 1152, 1153, . . . , 1155 included in the user-specific set of exercise classes 115. k At least one inertial sensor data 171, 172, 173, ..., 17 n , and the user-specific exercise classes 1151, 1152, 1153, ..., 115 k may each be associated with at least one class member 102A, 102B of an associated class 102, whereby the at least one class member 102A, 102B is selected based on the movement of the mobile treatment device 11. The associated class 102 may relate to a target surface, and the class members are zones of the target surface.
[0128] Thus, the motion pattern classification device 14 uses at least one inertial sensor data 171, 172, 173, ..., 17 n In order to classify the movements of the mobile treatment device 11 in a user-specific manner, the user-specific movement classes 1151, 1152, 1153, ..., 115 k The user may be configured to use a user-specific set 115 of
[0129] As shown in the examples of Figures 10 and 11, the apparatus 100 may include at least one associated class 102 for classifying the object in the second step 122. However, the apparatus 100 may include two or more associated classes in the second step 122, as shown in the example of Figure 9.
[0130] In the second step 122, for example, after a particular user is identified in the first step 121, different actions can be performed by the mobile treatment device 11. For example, the mobile treatment device 11 can change its operation mode based on the identified user. For example, the mobile treatment device 11 can be electrically driven and can include a motor, and the mobile treatment device 11 can change one or more motor specific characteristics, such as frequency, amplitude, or pulsation, based on the identified user. Additionally or alternatively, the mobile treatment device 11 can include one or more elements for communicating with or providing feedback to the user, for example, visual elements such as lights, e.g., LEDs, or tactile elements such as vibration motors. For example, the mobile treatment device 11 can change the user experience based on the identified user by changing the operation mode of the above elements for communicating, for example, by changing an LED light to a different color, or by providing different pulse feedback by a vibration motor based on the identified user.
[0131] In addition to or alternatively to identifying a particular user of a group of users, e.g., a family member of a family, the apparatus 100 may be configured to identify a particular user type. For example, if the mobile treatment device 11 is a toothbrush, some people may start brushing their teeth with their front teeth or incisors, while some other people may start brushing their teeth with their back teeth or molars. In a further example, if the personal appliance is a razor, some people may shave with the direction of hair growth, while some other people may shave against the direction of hair growth. In summary, a user type may be a type of user who uses the mobile treatment device 11 in a particular way. There may be two or more users that can be clustered into a group of user types. The user identification examples described above instead identify each user individually.
[0132] According to one embodiment for identifying a user type, at least one associated class 104 of the one or more associated classes 101, 102, 103, 104 may include at least two class members nA, nB, where the one associated class 104 may represent a user type of the mobile treatment device 11, a first class member nA may represent a first user type of the mobile treatment device 11, a second class member nB may represent a second user type of the mobile treatment device 11, and at least one motion class 151, 152, 153, ..., 154. k may be associated with either the first or second class members mA, mB for identifying a user type of the mobile treatment device 11 based on the movement of the mobile treatment device 11.
[0133] According to a further embodiment, the movement pattern classification device 14, after the step of identifying the user type, classifies two or more user type-specific movement classes 1151, 1152, 1153, ..., 1154 of the mobile treatment device 11, characterized by the identified user type. k The neural network 18 may be configured to select a user type specific set 115 including two or more exercise classes 151, 152, 153, ..., 154, ..., 155, after the step of identifying the user type, replace the set of exercise classes 15 with the selected user type specific set 115, and select two or more exercise classes 151, 152, 153, ..., 155, ..., 156, ..., 157, ..., 159, ..., 160, ..., 161, ..., 162, ..., 163, ..., 164, ..., 165, ..., 166, ..., 167, ..., 168, ..., 169, ..., 170, ..., 171, ..., 172, ..., 173, ..., 174, ..., 175, ..., 176, ..., 177, ..., 178, ..., 179, ..., 180, ..., 181, ..., 182, ..., 183, ..., 184, ..., 185, ..., 186, ..., 187, ..., 188, ..., 189, ..., 190, ..., 191, ..., 192, ..., 193, ..., 194, ..., 195, ..., 196, ..., 197, ..., 198, ..., 199, ..., 200, ..., 200, ..., 201, ..., 202, ..., 203, ..., 204, ..., 205, ..., 206, ..., 207, ..., 208, ..., 209, ..., 201, ..., 209, ..., 20 k , two or more user type-specific exercise classes 1151, 1152, 1153, ..., 115 k may be configured to replace
[0134] User-specific exercise classes 1151, 1152, 1153, ..., 115 k All that has been described above with respect to the user-specific set 115 of user type specific exercise classes 1151, 1152, 1153, ..., 115 k The same is true for the user type specific set 115.
[0135] As mentioned above, the identified user types may be clustered into clusters or groups of user types. Thus, the device 100 may perform a cluster analysis in which the user may use the mobile treatment device 11 a predefined number of times before being clustered into a particular user type group. For example, a user may use his / her razor five times in five days. On four of the five days, the user may shave against the grain of the beard. Thus, after the fifth day, the device 100 may cluster this user into a user type group in which all users shave against the grain of the beard.
[0136] The cluster analysis can also be performed at shorter time intervals. That is, the switching on and off of the toothbrush 11 can be performed directly in succession. For example, a user can switch his electric toothbrush 11 on and off at a first time and switch it on at a second time to resume the toothbrush 11 again. At the time of resuming the toothbrush 11, the device 100 of the present invention, in particular the neural network 18, can also be resumed. When the toothbrush 11 is switched on, it can collect information for the cluster analysis. However, at least the neural network 18 shall be resumed each time before new information for the cluster analysis is collected. In summary, the device 100 can perform the cluster analysis repeatedly (for example, five times) before finally clustering the users into a certain user type group.
[0137] After the users are clustered into groups specific to a particular user type, the neural network 18 generates user type specific movement patterns 1151, 1152, 1153, ..., 115 k , may use the associated user type specific presets 115.
[0138] According to such an embodiment, the movement pattern classification device 14 may be configured to repeatedly perform cluster analysis a predetermined number of times, and for each cluster analysis, the neural network 18 is restarted and, after restart, the neural network 18 is restarted by using at least one of the inertial sensor data 171, 172, 173, ..., 174. n , and at least one inertial sensor data 171, 172, 173, ..., 17 n , 151, 152, 153, ..., 155 included in the set of motion patterns 15. k , and the neural network 18 may be configured to select a user type specific movement pattern preset 115 after performing the cluster analysis a predefined number of times.
[0139] The apparatus 100 may allow even more scenarios for classifying the movement of the mobile treatment device 11. Therefore, reference is again made to FIG.
[0140] According to one embodiment, at least one class 102 of the one or more associated classes 101, 102, 103, 104 may include at least two class members 102A, 102B, where the one associated class 102 may represent a handling assessment of the mobile treatment device 11, a first class member 102A may represent a correct handling of the mobile treatment device 11, a second class member 102B may represent an incorrect handling of the mobile treatment device 11, and at least one classified motion class 151, 152, 153, ..., 154. k may be associated with either the first or second class member 102A, 102B for assessing the handling of the mobile treatment device 11 based on the motion of the mobile treatment device 11. According to one example, two or more motion classes 151, 152, 153, ..., 15 k may relate to the correct handling of the device, and these exercise classes may in turn be associated with the first class member 102A, and two or more exercise classes 151, 152, 153, ..., 15k may be related to incorrect handling of the device, and thus these exercise classes may be associated with the second class member 102B.
[0141] In other words, the apparatus 100 may be configured to verify whether a user of the mobile treatment device 11 is using the mobile treatment device 11 correctly or not. Of course, the one class 102 representing the handling evaluation can also be used as a class for the second step 122 of the two-step procedure described above in Figures 10 and 11, for example after identifying the user and / or the user type.
[0142] According to a further embodiment, at least one associated class 103 of the one or more associated classes 101, 102, 103, 104 may include at least two class members 103A, 103B, said one associated class 103 may represent the quality of the exercise performance of the mobile treatment device 11, a first class member 103A may represent a good exercise performance of the mobile treatment device 11 and a second class member 103B may represent a poor exercise performance of the mobile treatment device 11, and at least one classified exercise class 151, 152, 153, ..., 154 may represent a poor exercise performance of the mobile treatment device 11. k may be associated with either the first or second class members 103A, 103B for assessing the quality of the exercise execution of the mobile treatment device 11 based on the movement of the mobile treatment device 11.
[0143] In other words, the apparatus 100 may be configured to ascertain whether a user of the mobile treatment device 11 may use the mobile treatment device 11 in a good or bad manner. A good manner may be a manner of performing an intended mobile treatment device 11 movement, while a bad manner may be a manner of performing an unintended mobile treatment device 11 movement. For example, if the mobile treatment device 11 is a toothbrush, the apparatus may ascertain whether the user may have a good or bad brushing technique.
[0144] Of course, the above-mentioned one class 103 representing the quality of exercise execution can also be used as the associated class of the second step 122 of the two-step procedure described above in Figures 10 and 11, for example after identifying the user and / or user type.
[0145] Neural network 18 of device 100 may include the same or similar features as neural network 18 of device 10 described with reference to FIGS.
[0146] Everything that has been said above with respect to any feature of the neural network 18 of the apparatus 10 shown in Figures 4 to 7 also applies to the neural network 18 of the apparatus 100 described with reference to Figures 9 to 11.
[0147] FIG. 12 shows a block diagram of a method of the present invention for classifying the movement of a mobile treatment device 11 equipped with an inertial sensor 13 .
[0148] In block 1201, the method includes: determining whether or not the mobile treatment device 11 is moving in a set of motion patterns 15 based on two or more motion classes 151, 152, 153, ..., 154; k The method includes determining whether the
[0149] In block 1202, the method includes receiving at least one inertial sensor data 171, 172, 173, . . . , 17 from an inertial sensor 13. n providing the motion pattern classification device 14 with at least one inertial sensor data 171, 172, 173, . . . , 17 n represents the movement of the mobile treatment device 11.
[0150] In block 1203, the method includes computing, by the neural network 18, at least one inertial sensor data 171, 172, 173, . . . , 17 n , 152, 153, . . . , 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 1 kAt least one inertial sensor data 171, 172, 173, ..., 17 n , and classifying at least one classified motion class 151, 152, 153, ..., 15 k is associated with at least one class member 101A, 101B, 102A, 102B, ..., mA, mB of at least one associated class 101, 102, 103, 104, whereby at least one class member 101A, 101B, 102A, 102B, ..., mA, mB is selected based on the movement of the mobile treatment device 11.
[0151] According to yet a further example of the apparatus 10 , 100 of the present invention, the mobile treatment device 11 may be a personal instrument and the target surface 12 may be a body part to be treated by the mobile treatment device 11 .
[0152] According to yet a further example of the apparatus 10, 100 of the present invention, the mobile treatment device 11 or the mobile treatment device 11 may be provided with a pressure sensor for detecting the pressure applied to the target zone by the personal instrument and / or a load sensor for detecting the motor load of a motor that may drive the personal instrument.
[0153] The sensor data of each of the pressure sensors and / or load sensors is calculated based on at least one inertial sensor data 171, 172, 173, . . . , 17 n may additionally or alternatively be provided as an input to the neural unit 18.
[0154] According to yet a further example of the apparatus 10 of the present invention, the apparatus 10 is adapted to move one or more zones 211, 212, 213, ..., 214 of the target surface 12 on which the mobile treatment device 11 is located. m The device may include an output interface for outputting the information to a user.
[0155] According to yet a further example of the device 100 of the present invention, the device 100 may be provided with an output interface for outputting information to a user, said information relating to one or more associated classes 101, 102, 103, 104 and / or one or more class members 101A, 101B, ..., mA, mB of the one or more associated classes 101, 102, 103, 104.
[0156] In each of the embodiments described herein, the sensor data 171, 172, 173, ..., 17 n can be stored in the mobile treatment device or treatment device 11 and then provided to the apparatus 10, 100 in the manner described above. This stored sensor data 171, 172, 173, ..., 174 for the different zones or associated classes can be n Optional post-processing can be used to show the consumer or user on a dashboard how well and which zones they covered, what they missed, what was in and out of the target. This data can be shown per use and / or aggregated over time over two or more uses (i.e. showing the consumer or user a simple dashboard of how they have been brushing for the week).
[0157] Although some aspects are described in the context of an apparatus, it will be apparent that these aspects also represent a description of a corresponding method, with blocks or devices corresponding to method steps or features of method steps. Similarly, aspects described in the context of method steps also represent a description of a corresponding block or item or feature of a corresponding apparatus. Some or all of the method steps may be performed by (or using) a hardware apparatus, such as, for example, a microprocessor, a programmable computer, or electronic circuitry. In some embodiments, one or more of the most important method steps may be performed by such an apparatus.
[0158] Depending on the particular implementation requirements, embodiments of the present invention can be implemented in hardware, or in software, or at least partially in hardware, or at least partially in software. This implementation can be performed using a digital storage medium, such as a floppy disk, DVD, Blu-Ray, CD, ROM, PROM, EPROM, EEPROM, or FLASH memory, on which electronically readable control signals are stored and which cooperate (or can cooperate) with a programmable computer system so that the respective method is performed. The digital storage medium may therefore be computer readable.
[0159] Some embodiments according to the invention comprise a data carrier having an electronically readable control signal, the data carrier being capable of cooperating with a programmable computer system such that one of the methods described herein is performed.
[0160] Generally, embodiments of the present invention can be implemented as a computer program product having program code operable to perform one of these methods when the computer program product runs on a computer. The program code may for example be stored on a machine readable carrier.
[0161] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.
[0162] In other words, an embodiment of the inventive methods is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.
[0163] A further embodiment of the inventive method is therefore a data carrier (or digital storage medium, or computer readable medium) comprising the computer program for performing one of the methods described herein, having recorded thereon. The data carrier, digital storage medium, or recording medium is typically tangible and / or non-transitory.
[0164] A further embodiment of the inventive method is therefore a data stream or a sequence of signals representing a computer program for performing one of the methods described herein, the data stream or the sequence of signals being adapted for example to be transferred via a data communication connection, for example the Internet, Bluetooth Low Energy (BLE), WiFi or any kind of network, for example a mesh network.
[0165] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.
[0166] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.
[0167] Further embodiments according to the invention comprise an apparatus or a system configured to transfer (e.g. electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may for example be a computer, a mobile device, a memory device, etc. The apparatus or system may for example comprise a file server for transferring the computer program to the receiver.
[0168] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functionality of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by any hardware apparatus.
[0169] The apparatus described herein may be implemented using a hardware apparatus, a computer, or a combination of a hardware apparatus and a computer.
[0170] The methods described herein may be implemented using a hardware apparatus, a computer, or a combination of a hardware apparatus and a computer.
[0171] The above described embodiments are merely illustrative of the principles of the present invention. It is understood that modifications and variations to the arrangements and details described herein will be apparent to those skilled in the art. It is therefore the intention to be limited only by the scope of the following claims, and not by the specific details presented in the description and illustration of the embodiments described herein.
[0172] Additionally, dimensions and values disclosed herein should not be understood to be strictly limited to the exact numerical values recited. Instead, unless otherwise indicated, each such dimension is intended to mean both the recited value and a functionally equivalent range surrounding that value. For example, a dimension disclosed as "40 mm" is intended to mean "about 40 mm."
Claims
1. An apparatus (10, 100) for performing a localization of a mobile treatment device (11) relative to a target surface (12), the mobile treatment device (11) comprising at least one inertial sensor (13) and configured to treat the target surface (12), the apparatus (10) comprising: Two or more exercise classes (15) included in the set (15) of exercise classes of the mobile treatment device (11) 1 , 15 2 , 15 3 , …, 15 k a movement pattern classification device (14) configured to discriminate between At least one inertial sensor data (17) from the inertial sensor (13) 1 , 17 2 , 17 3 , …, 17 n an interface (16) for providing the at least one inertial sensor data (17) to the movement pattern classification device (14); 1 , 17 2 , 17 3 , …, 17 n and an interface (16) for representing the movement of the mobile treatment device (11); The motion pattern classification device (14) uses the at least one inertial sensor data (17 1 , 17 2 , 17 3 , …, 17 n ), and the exercise class (15) included in the set of exercise classes (15) 1 , 15 2 , 15 3 , …, 15 k ) to said at least one inertial sensor data (17 1 , 17 2 , 17 3 , …, 17 n ), wherein the motion classes (15) are classified by at least one neural network (18). 1 , 15 2 , 15 3 , …, 15 k ) each of which is arranged to define one or more different zones (21) relative to the target surface (12). 1 , 21 2 , 21 3 , …, 21 m ) is associated with (20 1 , 20 2 , 20 3 , …, 20 k ), thereby the exercise class (15 1 , 15 2 , 15 3 , …, 15 k ) the at least one inertial sensor data (17 1 , 17 2 , 17 3 , …, 17 n ) of the target surface (12) 1 , 21 2 , 21 3 , …, 21 m ) showing an estimate of the position of the mobile treatment device (11) relative to The mobile treatment device (11) is provided with a load sensor for detecting a motor load of a motor driving the mobile treatment device (11), and the interface (16) is adapted to receive the at least one inertial sensor data (17). 1 , 17 2 , 17 3 , …, 17 n ), the apparatus (10, 100) being arranged to provide at least one further sensor data from the load sensor to the movement pattern classification device (14).
2. The output (y t-1 , y t , y t+1 ) of the one or more zones (21) of the target surface (12). 1 , 21 2 , 21 3 , …, 21 m 2. The apparatus (10, 100) of claim 1, further comprising one or more probability or likelihood values for the estimation of the position of the mobile treatment device (11) relative to a target object.
3. The apparatus (10) is configured such that the mobile treatment device (11) t-1 , y t , y t+1 ) is located based on the highest probability or likelihood value of 3 3. The apparatus (10, 100) of claim 2, arranged to indicate
4. At least one exercise class (15) included in the set of exercise classes (15) NB ) in the outer zone (21) of the target surface (12) NB ) and associated with this exercise class (15 NB ) the at least one inertial sensor data (17 1 , 17 2 , 17 3 , …, 17 n ) is classified according to the fact that the mobile treatment device (11) is located in the outer zone (21) of the target surface (12). NB 4. The device (10, 100) according to any one of claims 1 to 3, wherein the device (10, 100) is located at
5. The apparatus (10, 100) of any one of claims 1 to 4, wherein the neural network (18) is a recurrent neural network, specifically a long short-term memory network or a gated recurrent unit network, more specifically, the recurrent neural network is a bidirectional recurrent neural network, and even more specifically, the recurrent neural network is an at least two-layer bidirectional recurrent neural network.
6. The interface (16) receives inertial sensor data (17) from the inertial sensor (13). 1 , 17 2 , 17 3 , …, 17 n ) is input to the movement pattern classification device (14) as 1 , X 2 , X 3 , …X n 6. The device (10, 100) according to any one of claims 1 to 5, arranged to provide as a
7. The apparatus (10) detects the zone (21) in which the mobile treatment device was located during a period related to the sequence of inertial sensor data based on a maximum or large criterion. 1 , 21 2 , 21 3 , …, 21 m 7. The apparatus (10) of claim 6, arranged to exhibit one or more of:
8. The at least one inertial sensor data (17 1 , 17 2 , 17 3 , …, 17 n 8. The apparatus (10) according to claim 1, wherein the inertial sensor data portion comprises at least one inertial sensor data portion, in particular at least three inertial sensor data portions of the group comprising: linear velocities in the x-, y- and z-directions, angular velocities about the x-, y- and z-axes, linear accelerations in the x-, y- and z-directions, and angular accelerations about the x-, y- and z-axes.
9. The apparatus (10) according to any one of the preceding claims, wherein the mobile treatment device (11) comprises a pressure sensor arranged to measure the pressure that the mobile treatment device (11) exerts on the target surface.
10. The motion pattern classification device (14) uses the at least one inertial sensor data (17 1 , 17 2 , 17 3 , …, 17 n ), the first neural network being adapted to receive a motion class (15) included in a first set of motion classes (15). 1 , 15 2 , 15 3 , …, 15 k ) to said at least one inertial sensor data (17 1 , 17 2 , 17 3 , …, 17 n ), the second neural network being configured to classify a motion class (15) included in a second set of motion classes (15). 1 , 15 2 , 15 3 , …, 15 k ) to said at least one inertial sensor data (17 1 , 17 2 , 17 3 , …, 17 n ) of the first and second sets of motion classes (15 1 , 15 2 , 15 3 , …, 15 k ) each of which is arranged to define one or more different zones (21) relative to the target surface (12). 1 , 21 2 , 21 3 , …, 21 m ) is associated with (20 1 , 20 2 , 20 3 , …, 20 k ), thereby the exercise class (15 1 , 15 2 , 15 3 , …, 15 k ) the at least one inertial sensor data (17 1 , 17 2 , 17 3 , …, 17 n ) of the target surface (12) 1 , 21 2 , 21 3 , …, 21 m ), and the output (y t-1 , y t , y t+1 ) and the output (y t-1 , y t , y t+1 ) of the one or more zones (21) of the target surface (12). 1 , 21 2 , 21 3 , …, 21 m The apparatus (10) according to any one of claims 1 to 9, used to provide an estimate of the position of the mobile treatment device (11) relative to a target object.
11. The apparatus (10) of any one of claims 1 to 10, wherein the neural network comprises a trained weight matrix and a bias vector.
12. The apparatus (10) according to any one of claims 1 to 11, wherein the mobile treatment device (11) is a personal instrument and the target surface (12) is a body part to be treated by the mobile treatment device (11).
13. The mobile treatment device (11) is an oral care device, the target surface (12) is an oral cavity, the oral cavity (12) is separated into a plurality of oral cavity zones (1a-9a, 1b-16b), and the movement class (15 1 , 15 2 , 15 3 , …, 15 k ) the at least one inertial sensor data (17 1 , 17 2 , 17 3 , …, 17 n 13. The apparatus (10) according to any one of claims 1 to 12, wherein the classification of (a) indicates an estimation of a position of the oral care device (11) relative to the one or more oral cavity zones (1a-9a, 1b-16b).
14. A method for performing a localization of a mobile treatment device (11) relative to a target surface (12), said mobile treatment device (11) comprising at least one inertial sensor (13) and configured to treat said target surface (12), said method comprising the steps of: Two or more exercise classes (15) included in the set (15) of exercise classes of the mobile treatment device (11) 1 , 15 2 , 15 3 , …, 15 k ) and At least one inertial sensor data (17) from the inertial sensor (13) 1 , 17 2 , 17 3 , …, 17 n ), wherein the at least one inertial sensor data (17) 1 , 17 2 , 17 3 , …, 17 n ) represents the movement of the movable treatment device (11); The at least one inertial sensor data (17) is processed by a neural network (18). 1 , 17 2 , 17 3 , …, 17 n ) and processing at least one motion class (15) included in the set of motion classes (15). 1 , 15 2 , 15 3 , …, 15 k ) to said at least one inertial sensor data (17 1 , 17 2 , 17 3 , …, 17 n and classifying the The exercise class (15 1 , 15 2 , 15 3 , …, 15 k ) each of which is located in one or more different zones (21) of the target surface (12). 1 , 21 2 , 21 3 , …, 21 n ), thereby associating said at least one exercise class (15 1 , 15 2 , 15 3 , …, 15 k The at least one inertial sensor data (17) 1 , 17 2 , 17 3 , …, 17 n ) of the target surface (12) 1 , 21 2 , 21 3 , …, 21 m ) showing an estimate of the position of the mobile treatment device (11) relative to The mobile treatment device (11) comprises a load sensor for detecting a motor load of a motor driving the mobile treatment device (11), and an interface (16) receives the at least one inertial sensor data (17). 1 , 17 2 , 17 3 , …, 17 n ), further arranged to provide at least one further sensor data from the load sensor to a movement pattern classification device (14).
15. A system comprising an apparatus (10) according to any one of claims 1 to 13 and a mobile treatment device (11) configured to treat the target surface (12) and comprising the at least one inertial sensor (13) and the load sensor.
Citation Information
Patent Citations
Failure diagnosis method
JP2002090267A
Methods and systems for locating oral cleaning devices
JP2018523507A
Determining the user's currently treated body part
JP2019500632A
Tooth brushing pattern analyzing system
KR1020130057688A
Toothbrush system with magnetometer for dental hygiene monitoring
WO2016180929A1
