Method of operating a wearable electronic device and electronic device

The described wearable device uses a wake-up gesture detection system with inertial and additional sensors to reduce power consumption and false detections, enhancing the user experience and battery life through sensor data fusion and machine learning.

WO2026234115A1PCT designated stage Publication Date: 2026-11-12AUSTRIAMICROSYSTEMS AG
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/EP2026/063139
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2025-05-08
Filing Date
2026-05-06
Publication Date
2026-11-12

AI Technical Summary

Technical Problem

Existing wearable electronic devices face challenges in achieving low-power operation and robust gesture detection to avoid false positives due to everyday motion, which impacts battery life and user experience.

Method used

A wearable electronic device equipped with an inertial sensor and additional sensors uses a wake-up gesture detection mechanism to activate gesture classification only when intended, reducing power consumption and false detections through a combination of sensor data fusion, post-processing, and machine learning models.

Benefits of technology

The solution enables a power-efficient and robust Human-Machine Interface (HMI) paradigm by minimizing false positives and extending battery life, ensuring seamless user interaction with wearable devices.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure EP2026063139_12112026_PF_FP_ABST
    Figure EP2026063139_12112026_PF_FP_ABST
Patent Text Reader

Abstract

A wearable device (1) comprises an inertial sensor (21), at least one further sensor (22) and a gesture detection system (23). The inertial sensor (21) and the further sensor (22) are designed to detect a movement of the wearable device (1). The gesture detection system (23) is designed to determine a gesture class (46) based on sensor data (41) provided by the inertial sensor (21) and / or the further sensor (22) for a gesture-based communication of a user (11) with the wearable device (1). A method (30) of operating the wearable device comprises the following method steps. It is checked whether the user (11) performed a wake-up gesture based on sensor data (41) provided by the inertial sensor (21). A gesture class (46) of a gesture of the user (11) is determined based on sensor data (41) provided by the further sensor (22) if the wake-up gesture is detected.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] 2024PF01549 1

[0002] METHOD OF OPERATING A WEARABLE ELECTRONIC DEVICE AND ELECTRONIC DEVICE

[0003] DESCRIPTION

[0004] The present invention refers to a method of operating a wearable electronic device and a wearable electronic device .

[0005] This patent application claims the priority of German patent application 10 2025 117 733.3, the disclosure content of whi-chis hereby incorporated by reference .

[0006] With the upcoming arrival of smart glasses and other wearable electronic devices, there is a need for a new Human-Machine Interface (HMI ) paradigm that combines low-power operation for all-day use with social acceptability, such as avoiding voice commands or exaggerated hand gestures . From the state of the art, it is known to use different sensors such as inertial sensors to detect gestures, where a gesture sensing solution is always running to check the motion generated artifacts and correlate them to specific finger or wrist movements . One issue with such a solution is that it indeed needs to run all day, opening to the risk of false detections due to everyday motion or to the fact that it can speed up the power consumption of the wearable device .

[0007] An obj ective of the present invention is to specify an improved method of operating a wearable electronic device and to provide an improved wearable electronic device . This obj ective is solved by a method of operating a wearable electronic device and a wearable electronic device comprising the features of the respective independent claims . Advantageous embodiments are specified in the dependent claims, respectively.

[0008] A wearable electronic device comprises an inertial sensor, at least one further sensor and a gesture detection system. The inertial sensor and the at least one further sensor are de-2024PF01549 2

[0009] signed to detect a movement of the wearable electronic device . The gesture detection system is designed to determine a gesture class based on sensor data provided by the inertial sensor and / or the at least one further sensor for a gesturebased communication of a user of the wearable electronic device with the wearable electronic device according to a predefined gesture vocabulary comprising a plurality of gesture classes . A method of operating the wearable electronic device comprises the following method steps . It is checked whether the user of the wearable electronic device performed a wakeup gesture based on sensor data provided by the inertial sensor . A gesture class of a gesture of the user is determined based on sensor data provided by the at least one further sensor or based on sensor data provided by the inertial sensor and the at least one further sensor if the wake-up gesture is detected.

[0010] The wearable electronic device is also called wearable device briefly in the following description. The wearable device can e . g. be designed as smart glasses, a smartwatch, a wristband, a smart ring or as a head-mountable display such as an augmented or virtual reality (AR / VR) display. The term wearable means that the wearable device is attachable to a body part of the user, e . g. to a wrist of the user . Instead of attaching the wearable device to the wrist it could also be attached to an ankle or to any other body part of the user . However, the term wearable device shall also include handheld devices . For example, the handheld wearable device can be designed as a controller unit . Especially in AR / VR applications, a controller can be used in combination with a head-mountable display. However, in the case of an attachable wearable device hands free operations enable a more seamless interaction with both the user interface and the environment in mixed reality.

[0011] In order to translate movements of the wearable device into gestures enabling a HMI paradigm, i . e . a communication of the user with the wearable device according to the pre-defined2024PF01549 3

[0012] gesture vocabulary, the wearable device comprises the inertial sensor and at least one further sensor, both of which are designed to detect a movement of the wearable device .

[0013] The wearable device can be worn and used all day long. This poses several challenges . The gesture detection system of the wearable device needs to be operable with a low power to have a reduced impact to a battery of the wearable device that needs to last as long as possible . The gesture detection system also needs to be robust to everyday motion and gestures of the user as the wearable device often is meant to be worn in everyday life . The user can perform all kinds of everyday movements and gestures, not only the movements according to the defined gesture vocabulary.

[0014] The inertial sensor is part of a wake-up system using only acceleration / motion sensor data of the inertial sensor . The inertial sensor can be designed as a translational and / or as a rotational sensor, i . e . a gyroscope, comprising at least one translational acceleration sensor channel and / or at least one rotational acceleration sensor channel . For the inertial sensor, the different channels can be assigned to different translational directions and / or rotational axes, respectively. Thus, the inertial sensor can be designed to detect translational and / or rotational movement of the wearable device .

[0015] A wake-up gesture detection that uses only the sensor data of the inertial sensor as input is performed by checking if specific conditions are verified in order classify a gesture of the user as the wake-up gesture . The wake-up gesture is a gesture class that is defined in the gesture vocabulary and can be any desired gesture . One example of a wake-up gesture is keeping a hand and the wearable device attached to the wrist in a certain position for a pre-defined time span. Generally, the wake-up gesture should be a specific, non-complex gesture that is reliably detectable by only using the sensor data provide by the inertial sensor . Since only sensor data2024PF01549 4

[0016] provided by the inertial sensor is used to detect the wakeup gesture, the gesture detection system of the wearable device advantageously comprises a low power consumption.

[0017] Furthermore, the overall power consumption of the gesture detection system and thus the wearable device is reduced since the gesture class of the user' s gesture is only determined based on sensor data provided by the at least one further sensor or based on sensor data provided by the inertial sensor and the at least one further sensor if the wake-up gesture has been detected based on the sensor data provided by the inertial sensor . This means that the gesture detection system is only activated for the determination of the gesture class of the gesture if the user wants to interact with the wearable device . Moreover, the wake-up gesture detection reduces the risk of activating the gesture detection system when not intended.

[0018] Advantageously, this also reduces false detections which occur when a gesture class has been falsely attributed to another gesture or to some arbitrary movement not defined in the gesture vocabulary. False positives would occur significantly more frequent when the gesture detection system would determine gesture classes based on sensor data provided by the further sensor continuously. These false detections can be interpreted as valid user commands, thereby causing a lot of friction in the user experience . The wake-up gesture detection reduces the risk of false positives significantly. Thus, the present method advantageously enables a new HMI paradigm for wearable devices that is power efficient and more robust to everyday motion.

[0019] In an embodiment the at least one further sensor is designed as a further inertial sensor, an optical sensor, particularly as a photoplethysmography sensor, a mechanical sensor, a myoelectric sensor, a biosensor, a camera, a light detection and ranging (LIDAR) sensor, a time-of-f light sensor or a capacitive sensor .2024PF01549 5

[0020] Preferably, the further sensor is designed as a photoplethysmography (PPG) sensor . PPG is based on the principle of determining changes of the blood flow in arteries of the skin of the user by measuring the absorption of electromagnetic radiation emitted onto the skin. A wavelength dependent absorption takes place due to the presence of haemoglobin in blood. Thus, a pulse rate of the blood stream can be determined, since the volume of the blood flow changes with the heart rate . However, sensor data provided by the PPG sensor can also be used to determine movements and gestures which can be assigned to specific signal patterns of the PPG signal .

[0021] A PPG sensor comprises at least one light emitting diode (LED) designed to emit electromagnetic radiation from a desired wavelength spectrum and at least one photodiode (PD) designed to detect electromagnetic radiation which has been emitted by the LED and reflected at the skin of the user . However, similar to multiple possible axes, i . e . channels of the inertial sensor, the PPG sensor can also comprise multiple channels for different wavelength spectra . Apart from LEDs, the PPG sensor can alternatively or additionally comprise at least one laser diode, e . g. a laser diode designed to emit electromagnetic radiation from the infrared spectral range . As an example, the PPG sensor can comprise three channels, a first channel for electromagnetic radiation from the green spectral range, a second channel for electromagnetic radiation from the blue spectral range and a third channel for electromagnetic radiation from the red or infrared spectral range . For each channel, an LED and a PD can be provided, however, it is also possible that a plurality of channels share a common PD, i . e . a PD is provided for at least two LEDs, as an example .

[0022] Determining the gesture class can be based on multiple sensor channels of the at least one further sensor . Additionally, sensor data from multiple channels of the inertial sensor can2024PF01549 6

[0023] be used to determine the gesture class . In other words, a plurality of sensor channel signals can be fused or merged when determining the gesture class, wherein at least sensor data of the further sensor is used. This can reduce the detection of false positives, advantageously.

[0024] The wearable device can comprise a plurality of further sensors . Each of the further sensors can be designed as a further inertial sensor, an optical sensor, particularly as a photoplethysmography sensor, a mechanical sensor, a myoelectric sensor, a biosensor, a camera, a LIDAR sensor, a time-of-flight sensor or a capacitive sensor . Any one of these sensors can be used to detect the movement of the wearable device based on gesture specific signal patterns, as an example . Note, that a gesture can be based on device control by means of a capacitive sensor or a force sensor of the wearable device . The drawback here is that gestures are limited to very basic interactions, basically to either a push or a slide gesture .

[0025] It is also possible to detect the movement of the user by a camera . In this case, a so-called remote hand tracking can be performed, which is usually done by using several active or passive cameras, e . g. arranged outside an AR / VR device . Alternatively or additionally, it is possible to use a depth sensor like a time-of-f light sensor or a LIDAR-sensor for the remote hand tracking.

[0026] In an embodiment sensor data of the inertial sensor and the at least one further sensor is acquired continuously. Alternatively, sensor data of the inertial sensor is acquired continuously and sensor data of the at least one further sensor is acquired only if the wake-up gesture is detected.

[0027] As the gesture detection system is designed to detect the wake-up gesture based only on sensor data provided by the inertial sensor as input, a data acquisition device of the wearable device is designed to collect sensor data only pro-2024PF01549 7

[0028] vided by the inertial sensor . Then, only if the wake-up gesture is detected, sensor data from the at least one further sensor is acquired, advantageously reducing even more the power consumption of the wearable device . The sensor data provided by the inertial sensor can be acquired as soon as the wearable device is attached to the user or held by the user . Optionally, the wearable device can be activated before acquiring sensor data from the inertial sensor and checking whether the wake-up gesture is performed.

[0029] In an embodiment gesture classes are determined for multiple consecutive gestures performed by the user . Sensor signals of the multiple consecutive gestures are checked with respect to temporal consistency. At least one determined gesture class is not considered for the communication between the user and the wearable electronic device if its sensor signal is temporally inconsistent . Alternatively or additionally, different gesture classes are summarized in one common gesture class if the sensor signals associated with the different gesture classes are temporally consistent .

[0030] This post-processing step helps to minimizes further the risk of false detections to ensure temporally smooth and consistent outputs . This further ensures an uninterrupted user experience . Post-processing takes as input gesture classes associated with several consecutive gestures and ensures that they are temporally consistent . For example, a gesture can be performed within a certain typical time span, e . g. within 0.8-lsec . In case of an exemplary stride of 100ms for the gesture class determination, several consecutive results can correspond to the same gesture . If one of the determination results is deviating from the other determination results, a temporal inconsistency is detected. Thus, only determination results comprising more votes can be considered for the communication of the user with the wearable device .

[0031] If the user performs multiple gestures, a false positive gesture detection can be removed if the corresponding sensor2024PF01549 8

[0032] signal is inconsistent with other sensor signals . Removing of a false positive means that the determined gesture class is not considered for the communication between the user and the wearable device, i . e . the falsely detected gesture is not used as a command, for example . Advantageously, the problem of the false positives is significantly mitigated by the post-processing step which helps to remove arbitrary predictions that are not temporally consistent .

[0033] On the other hand, it may be that sensor signals of multiple gestures are temporally consistent while different gesture classes have been determined for them. In this case, it can be convenient to summarize the different gesture classes determined for temporally consistent sensor signals in one common gesture class, i . e . one gesture class is determined for the temporally consistent sensor signals . When checking whether the sensor signals of the multiple consecutive gestures are temporally consistent sensor data provided by the at least one further sensor is used. Additionally, sensor data provided by the inertial sensor can be used.

[0034] In an embodiment a start time and an end time of the gesture is determined based on sensor data provided by the inertial sensor and / or the at least one further sensor . The gesture class is determined based on sensor data acquired between the start and the end time .

[0035] The start and end time of the gesture can be determined by the gesture detection system. However, the determination of the gesture class and the determination of the start and the end time of the gesture may be performed separately, e . g. by different models stored on the gesture detection system. Advantageously, the gesture detection system can determine the gesture class with less data input . Also, the determination of the start time and the end time of the gesture can be used when checking sensor signals with respect to temporal consistency .2024PF01549 9

[0036] In an embodiment determining the gesture class is performed by a machine learning (ML) model, particularly by artificial intelligence (Al ) . Particularly, the determination of the gesture class can be performed by a neural network. Similarly, the determination of the start and the end time of the gesture can be performed by a further machine learning model, particularly by artificial intelligence, e . g. a further neural network. However, a single ML model can be used to determine the gesture class and the start and end time of the gesture . The ML model is designed to use sensor data of the at least one further sensor and optionally sensor data provided by the inertial sensor in addition as an input, wherein multiple channels of the respective sensors can be used as input . As an output, the ML model generates a gesture class, which also can be called a gesture ID.

[0037] To train the machine learning model the gesture vocabulary has to be defined first . Second, a dataset comprising sensor data of a plurality of the gestures defined in the gesture vocabulary has to be acquired. This data set in combination with the corresponding gesture classes builds a training data set for the ML model . Preferably, the acquiring of the training data set can be performed with different users in order to account for user-to-user variations of the performed gestures . The training data set can also be extended by adding a personalized training data set comprising sensor data of gestures and corresponding gesture classes performed by a certain user . If a neural network is used as a ML model to determine the gesture class, the neural network can be designed as a convolutional neural network (CNN) , as an example . However, a support vector machine (SVM) can be used instead of a neural network to determine the gesture class, for example .

[0038] In an embodiment determining the gesture class is performed in two steps . In a first step, a first gesture class is determined by a shallow neural network. The first gesture class is either a class corresponding to no gesture or a class corresponding to some gesture . In a second step, the gesture2024PF01549 10

[0039] class is determined by a deep neural network if the first gesture class corresponds to some gesture .

[0040] A number of neuron layers of the shallow neural network and a number of the neuron layers of the deep neural network are not limited to a specified number, respectively. However, the deep neural network comprises more neuron layers than the shallow neural network. In this case, the shallow neural network is designed to discriminate between no gesture and any gesture and then, in case of an "any gesture" prediction, the deep neural network is designed to determine the gesture class . This two-step approach can increase an overall accuracy of the gesture detection system.

[0041] Instead of using a shallow neural network and a deep neural network for the two-step approach, any other ML models can be used which differ in complexity, e . g. a simple ML model can be used to discriminate between no gesture and any gesture, while a more complex ML model is used to determine the gesture class . Instead of using a machine learning model for the determination of the gesture class, a signal processing solution based on signal statistics and heuristics can be used, alternatively. Similarly, a two-step approach can be used, wherein solutions comprising different complexity can be used to differentiate between the detection of no gesture and some gesture .

[0042] In an embodiment the gesture class is determined based on sensor data provided by the at least one further sensor or sensor data provided by the inertial sensor and the at least one further sensor acquired for previously performed gestures . Advantageously, by also considering sensor data acquired for previously performed gestures, a temporal consistency can be ensured on the ML-model level .

[0043] A wearable electronic device comprises an inertial sensor, at least one further sensor and a gesture detection system. The inertial sensor and the at least one further sensor are de-2024PF01549 11

[0044] signed to detect a movement of the wearable electronic device for a gesture-based communication of a user of the wearable electronic device with the wearable electronic device according to a pre-defined gesture vocabulary comprising a plurality of gesture classes . The gesture detection system is designed to determine a gesture class based on sensor data provided by the inertial sensor and / or the at least one further sensor . The gesture detection system is designed to check whether the user of the wearable electronic device performed a wake-up gesture based on sensor data provided by the inertial sensor and to determine a gesture class of a gesture of the user based on sensor data provided by the at least one further sensor or based on sensor data provided by the inertial sensor and the at least one further sensor if the wakeup gesture is detected. Advantageously, the wearable device can be operated such that the power consumption is reduced since the gesture detection system is designed to determine the gesture class only if the wake-up gesture is detected, first . Furthermore, this wake-up system reduces the detection of false positives .

[0045] In an embodiment the gesture detection system is designed to acquire sensor data of the inertial sensor continuously and sensor data of the at least one further sensor only if the wake-up gesture is detected. Advantageously, the power consumption of the wearable device is further reduced. However, the sensor data of the at least one further sensor can be acquired also continuously.

[0046] In an embodiment the gesture detection system is designed to determine gesture classes of multiple consecutive gestures performed by the user and to check sensor signals of multiple consecutive gestures with respect to temporal consistency and to consider only gesture classes corresponding to temporally consistent sensor signals for the communication between the user and the wearable electronic device . Alternatively or additionally, the gesture detection system is designed to summarize different gesture classes in one common gesture class2024PF01549 12

[0047] if the sensor signals associated with the different gesture classes are temporally consistent . This post processing approach advantageously further reduces the detection of false positives .

[0048] In an embodiment gesture detection system is designed to run a machine learning model, particularly an artificial intelligence, particularly a neural network, designed to determine the gesture class .

[0049] In an embodiment the gesture detection system is designed to run a shallow neural network and a deep neural network. The shallow neural network is designed to determine a first gesture class . The first gesture class is either a class corresponding to no gesture or a class corresponding to some gesture . The deep neural network is designed to determine the gesture class if the shallow neural network has determined that the first gesture class corresponds to some gesture .

[0050] The above-described properties, features and advantages of this invention and the way in which they are achieved will become clearer and more clearly understood in association with the following description of the exemplary embodiments which are explained in greater detail in association with the drawings . Here in schematic illustration in each case :

[0051] Fig. 1 : different wearable devices;

[0052] Fig. 2 : a wearable device according to the present invention;

[0053] Fig. 3 : a method of operating the wearable device of Fig. 2 ; and

[0054] Fig. 4 : a two-step approach of a gesture detection.

[0055] Fig. 1 schematically shows different wearable devices 1 that can be attached to different body parts of a user 11.2024PF01549 13

[0056] As an example, a wearable device 1 can be designed as smart glasses 2 that can be worn by the user 11. Apart from that, the wearable device 1 can be designed as headphones 3, for example in-ear headphones 3. The wearable device 1 can also be designed as a wrist band 4, a smart watch 5 or a smart ring 6, which can be attached to the wrist or a finger of the user 11, respectively.

[0057] The wearable device 1 can also be designed as some device which is not shown in Fig. 1. For example, the wearable device can be designed as a handheld controller . Alternatively, the wearable device 1 can be designed as a head mountable display such as an AR / VR display. At least one controller and the head-mountable can be used in combination, too . Furthermore, the wearable device 1 can be integrated in clothing, e . g. in a shoe or in a t-shirt .

[0058] The wearable device 1 is designed for a gesture-based communication with the user 11, which means that the user 11 can interact with the wearable device 1 attached to a body part of the user 11 or held by the user 11 by performing specific gestures which are defined in a gesture vocabulary, i . e . the gesture vocabulary comprises different gesture classes .

[0059] Fig. 2 schematically shows more details of the wearable device 1. The wearable device 1 comprises an inertial sensor 21, at least one further sensor 22 and a gesture detection system 23. The inertial sensor 21 and the at least one further sensor 22 are designed to detect a movement of the wearable device 1. The wearable device 1 can comprise any number of further sensors 22.

[0060] The gesture detection system 23 is designed to determine a gesture class based on sensor data provided by the inertial sensor 21 and / or the at least one further sensor 22 for the gesture-based communication of the user 11 with the wearable device 1. The inertial sensor 21 can be designed as a trans-2024PF01549 14

[0061] lational acceleration sensor or as a rotational acceleration sensor (gyroscope) .

[0062] The further sensor 22 can be designed as any sensor which is designed to provide sensor data which can be correlated with a movement and / or an acceleration of the wearable device 1. For example, the further sensor 22 can be designed as a further inertial sensor . Alternatively, the further sensor 22 can be designed as an optical sensor, particularly as a photoplethysmography sensor, a mechanical sensor, a myoelectric sensor, a biosensor, a camera, a LIDAR sensor, a time-of-flight sensor or a capacitive sensor . If the wearable device 1 comprises a plurality of further sensors 22, each of the further sensors 22 can be designed as one of the specified sensors . Preferably, the at least one further sensor is designed as a PPG-sensor .

[0063] As already explained, the inertial sensor 21 and the at least one further sensor 22 can comprise multiple sensor channels, respectively. For example, the inertial sensor 21 can comprise three channels for three different acceleration directions, whether translational or rotational . Also, a PPG-sensor can comprise different channels for different wavelength spectra . In this case, the PPG-sensor comprises a plurality of light sources and light detectors for the different electromagnetic spectra .

[0064] Since the wearable device 1 can be worn all-day, it is necessary to provide a low power consuming system. The wearable device 1 of Fig. 2 is based on the idea to reduce the power consumption by introducing a wake-up mechanism enabling the gesture-based communication only if the user 11 wishes to communicate with the wearable device 1 .

[0065] Therefore, the gesture detection system 23 is designed to check whether the user 11 performs a wake-up gesture based on sensor data provided by the inertial sensor 21. The wake-up gesture can be any gesture defined as the wake-up gesture in2024PF01549 15

[0066] the gesture vocabulary. Furthermore, the gesture detection system 23 is designed to determine the gesture class of a gesture of the user based on sensor data provided by the at least one further sensor 22 or based on sensor data provided by the inertial sensor 21 and the at least one further sensor 22 if the wake-up gesture is detected.

[0067] The gesture detection system 23 is designed to determine the gesture class based on sensor data of the further sensor 22, i . e . sensor data of the further sensor 22 is used in any case . Additionally, the gesture detection system 23 can be designed to determine the gesture class also based on sensor data of the inertial sensor 21. However, since the sensor data of the further sensor 22 is only evaluated if the wake-up gesture is detected, the power consumption of the gesture detection system 23 is reduced. Additionally, the gesture detection system 23 can be designed to acquire sensor data of the at least one further sensor 22 only if the wake-up gesture is detected, i . e . that the further sensor 22 can be activated only if the wake-up gesture has been detected based only on sensor data provided by the inertial sensor 21. This reduces the power consumption even further . In contrast, sensor data of the inertial sensor 21 can be acquired continuously, after the wearable device 1 is attached to some body part of the user 11.

[0068] For the determination of the gesture class the gesture detection system 23 is designed to run a machine learning model, particularly an artificial intelligence, preferably a neural network. To run the ML-model the gesture detection system 23 can comprise a storage with data representing the trained ML-model, e . g. this data can comprise weighted coefficients for neuron connections . The storage does not necessarily need to be a part of the gesture detection system 23, it can be also designed separately from the gesture detection system 23 and be connected to the gesture detection system 23.2024PF01549 16

[0069] Fig. 3 schematically shows a method 30 of operating the wearable device of Fig. 2. In a first method step 31, it is checked whether the user 11 of the wearable device 1 performs a wake-up gesture based on sensor data provided by the inertial sensor 21. In a second method step 32, A gesture class of a gesture of the user 11 is determined based on sensor data provided by the at least one further sensor 22 or based on sensor data provided by the inertial sensor 21 and the at least one further sensor 22 if the wake-up gesture is detected .

[0070] Apart from a lower power consumption, a detection of false positives can be also reduced by running the ML-model for the determination of the gesture class only if the wake-up gesture is detected. In addition, a post-processing can reduce the detection of false positives even further .

[0071] Post-processing can be performed by determining gesture classes for multiple consecutive gestures performed by the user 11 and checking sensor signals of the multiple consecutive gestures with respect to temporal consistency. At least one determined gesture class can be dismissed and not considered for the communication between the user 11 and the wearable device 1 if its sensor signal is temporally inconsistent . Alternatively or in addition, different gesture classes can be summarized in one common gesture class if the sensor signals associated with the different gesture classes are temporally consistent .

[0072] In order to prove the temporal consistency, it may be convenient to determine a start time and an end time of the gesture based on sensor data provided by the inertial 21 sensor and / or the at least one further sensor 22. In this case, it is possible to determine the gesture class solely based on sensor data acquired between the start and the end time of a gesture which can reduce the power consumption even more .2024PF01549 17

[0073] Fig. 4 schematically shows a two-step approach of determining the gesture class 46. In other words, Fig. 4 shows exemplary details of the second method step 32 of Fig. 3, i . e . after the wake-up gesture has already been detected.

[0074] At first, sensor data 41 of the further sensor 22 is acquired, e . g. sensor data of the PPG-sensor . Additionally, sensor data 41 of the inertial sensor 21 can be acquired, too for the determination of the gesture class 46. Before determining the gesture class 46, a pre-processing of the acquired sensor data 41 can be performed, thereby providing pre-processed sensor data 42. Pre-processing can include normalization, frequency-based filtering, detrend, etc . To ensure that the input data of the ML-model are uniform, but at the same time signal spikes generated by the movement of the wearable device 1 are preserved, the data pre-processing can be an essential step .

[0075] Pre-processing of the sensor data 41 of the at least one further sensor 22 or the inertial sensor 21 and the at least one further sensor 22 may also include a selection of at least one sensor channel which shall be used as input . Apart from that, it is possible to choose sensor data 41 provided by specific further sensors 22, thereby enabling a fusion of sensor data of multiple further sensors 22. The sensor data 42 can also be truncated in order to obtain a certain data window. Also, specific signal processing manipulation techniques can be applied, such as a high frequency or a low frequency filter . However, the pre-processing can also be omitted .

[0076] The sensor data 41 or the pre-processed sensor data 42 is provided to a first ML-model 43. The first ML-model 43 can be designed as an artificial intelligence, e . g. a neural network, particularly a shallow neural network. In a first step of the two-step approach, a first gesture class 44 is determined by the first ML-model 43. The first gesture 44 class is either a class corresponding to no gesture or a class corre-2024PF01549 18

[0077] spending to some gesture . As a first result, the first ML-model 43 just provides an information about whether some gesture has been determined or not . If no gesture has been determined, sensor data 41 of the at last one further sensor 22 can be acquired again.

[0078] If it has been determined that some gesture has been detected, this information is provided to a second ML-model 45. The second ML-model 45 can be designed as an artificial intelligence, e . g. a neural network, particularly a deep neural network. The second ML-model 45 is designed to determine the gesture class 46 of the gesture based on the acquired sensor data 41 or the pre-processed sensor data 42. As a final result, the second ML-model 45 provides the gesture class 46, which can be, e . g. a pinch, a double-pinch, the closing of a hand, a thumbs-up gesture or a swipe gesture, e . g. an inward swipe, an outward swipe, an up swipe or a down swipe . Generally, the gesture class can also be determined by only using one ML-model instead of two ML-models 43, 45. For example, the first ML-model 43 can be omitted.

[0079] The invention has been illustrated and described in detail with the aid of the preferred exemplary embodiments . Nevertheless, the invention is not restricted to the examples disclosed. Rather, other variants may be derived therefrom by a person skilled in the art without departing from the protective scope of the invention.2024PF01549 19

[0080] REFERENCE SYMBOLS

[0081] 1 Wearable electronic device

[0082] 2 smart glasses

[0083] 3 headphones

[0084] 4 wrist band

[0085] 5 smart watch

[0086] 6 smart ring

[0087] 11 user

[0088] 21 inertial sensor

[0089] 22 further sensor

[0090] 23 gesture detection system

[0091] 30 method of operating a wearable electronic device 31 first method step

[0092] 32 second method step

[0093] 41 sensor data

[0094] 42 pre-processed sensor data

[0095] 43 first machine learning model

[0096] 44 first gesture class

[0097] 45 second machine learning model

[0098] 46 gesture class

Claims

1. 2024PF01549 20CLAIMS1 . Method ( 30 ) of operating a wearable electronic device ( 1 ) , wherein the wearable electronic device ( 1 ) comprises an inertial sensor ( 21 ) , at least one further sensor ( 22 ) and a gesture detection system ( 23 ) ,wherein the inertial sensor ( 21 ) and the at least one further sensor ( 22 ) are designed to detect a movement of the wearable electronic device ( 1 ) ,wherein the gesture detection system ( 23 ) is designed to determine a gesture class ( 46 ) based on sensor data ( 41 ) provided by the inertial sensor ( 21 ) and / or the at least one further sensor ( 22 ) for a gesture-based communication of a user ( 11 ) of the wearable electronic device ( 1 ) with the wearable electronic device ( 1 ) according to a pre-defined gesture vocabulary comprising a plurality of gesture classes ( 46 ) ,wherein the method ( 30 ) comprises the following method steps ( 31 , 32 ) :- checking whether the user ( 11 ) of the wearable electronic device ( 1 ) performed a wake-up gesture based on sensor data ( 41 ) provided by the inertial sensor ( 21 ) ,- determining a gesture class ( 46 ) of a gesture of the user ( 11 ) based on sensor data ( 41 ) provided by the at least one further sensor ( 22 ) or based on sensor data ( 41 ) provided by the inertial sensor ( 21 ) and the at least one further sensor ( 22 ) i f the wake-up gesture is detected .2 . The method ( 30 ) according to claim 1 ,wherein the at least one further sensor ( 22 ) is designed as a further inertial sensor, an optical sensor, particularly as a photoplethysmography sensor, a mechanical sensor, a myoelectric sensor, a biosensor, a camera, a LIDAR sensor, a time-of- flight sensor or a capacitive sensor .3 . The method ( 30 ) according to claim 1 or 2 ,wherein sensor data ( 41 ) of the inertial sensor ( 21 ) is acquired continuously and sensor data ( 41 ) of the at least one2024PF01549 21further sensor (22 ) is acquired only if the wake-up gesture is detected.

4. The method (30) according to one of the preceding claims, wherein gesture classes (46) are determined for multiple consecutive gestures performed by the user ( 11 ) ,wherein sensor signals of the multiple consecutive gestures are checked with respect to temporal consistency,wherein at least one determined gesture class (46) is not considered for the communication between the user ( 11 ) and the wearable electronic device ( 1 ) if its sensor signal is temporally inconsistent and / or,wherein different gesture classes (46) are summarized in one common gesture class if the sensor signals associated with the different gesture classes (46) are temporally consistent .

5. The method (30) according to one of the preceding claims, wherein a start time and an end time of the gesture is determined based on sensor data provided by the inertial sensor (21 ) and / or the at least one further sensor (22 ) ,wherein the gesture class (46) is determined based on sensor data (41 ) acquired between the start and the end time .

6. The method (30) according to one of the preceding claims, wherein determining the gesture class (46) is performed by a machine learning model, particularly by artificial intelligence, particularly by a neural network.

7. The method (30) according to claim 6,wherein determining the gesture class (46) is performed in two steps,wherein in a first step, a first gesture class (44 ) is determined by a shallow neural network (43) ,wherein the first gesture class (44 ) is either a class corresponding to no gesture or a class corresponding to some gesture,2024PF01549 22wherein in a second step, the gesture class ( 46 ) is determined by a deep neural network ( 45 ) i f the first gesture class ( 44 ) corresponds to some gesture .8 . The method ( 30 ) according to one of the preceding claims , wherein the gesture class ( 46 ) is determined based on sensor data ( 41 ) provided by the at least one further sensor ( 22 ) or sensor data ( 41 ) provided by the inertial sensor ( 21 ) and the at least one further sensor ( 22 ) acquired for previously performed gestures .9 . Wearable electronic device ( 1 )comprising an inertial sensor ( 21 ) , at least one further sensor ( 22 ) and a gesture detection system ( 23 ) ,wherein the inertial sensor ( 21 ) and the at least one further sensor ( 22 ) are designed to detect a movement of the wearable electronic device ( 11 ) for a gesture-based communication of a user ( 11 ) of the wearable electronic device ( 1 ) with the wearable electronic device ( 1 ) according to a pre-defined gesture vocabulary comprising a plurality of gesture classes ( 46 ) ,wherein the gesture detection system is designed to determine a gesture class ( 46 ) based on sensor data provided by the inertial sensor ( 21 ) and / or the at least one further sensor ( 22 ) ,wherein the gesture detection system ( 23 ) is designed to check whether the user ( 11 ) of the wearable electronic device ( 1 ) performed a wake-up gesture based on sensor data ( 41 ) provided by the inertial sensor ( 21 ) and to determine a gesture class ( 46 ) of a gesture of the user ( 11 ) based on sensor data ( 41 ) provided by the at least one further sensor ( 22 ) or based on sensor data provided by the inertial sensor ( 21 ) and the at least one further sensor ( 22 ) i f the wake-up gesture is detected .10 . The wearable electronic device ( 1 ) according to claim 9 , wherein the gesture detection system ( 23 ) is designed to acquire sensor data ( 41 ) of the inertial sensor ( 21 ) continu-2024PF01549 23ously and sensor data (41 ) of the at least one further sensor (22 ) only if the wake-up gesture is detected.

11. The wearable electronic device ( 1 ) according to claim 9 or 10,wherein the gesture detection system (23) is designed to determine gesture classes (46) of multiple consecutive gestures performed by the user ( 11 ) and to check sensor signals of multiple consecutive gestures with respect to temporal consistency and to consider only gesture classes (46) corresponding to temporally consistent sensor signals for the communication between the user ( 11 ) and the wearable electronic device ( 1 ) and / or,wherein the gesture detection system (23) is designed to summarize different gesture classes (46) in one common gesture class if the sensor signals associated with the different gesture classes (46) are temporally consistent .

12. The wearable electronic device ( 1 ) according to one of the claims 9 to 11,wherein the gesture detection system (23) is designed to run a machine learning model, particularly an artificial intelligence, particularly a neural network, designed to determine the gesture class (46) .

13. The wearable electronic device ( 1 ) according to claim 12, wherein the gesture detection system (21 ) is designed to run a shallow neural network (43) and a deep neural network (45) , wherein the shallow neural network (43) is designed to determine a first gesture class (44 ) ,wherein the first gesture class (44 ) is either a class corresponding to no gesture or a class corresponding to some gesture,wherein the deep neural network (45) is designed to determine the gesture class (46) if the shallow neural network (43) has determined that the first gesture class (44 ) corresponds to some gesture .