Device and method for recognizing head gesture

Abstract

The present invention relates to a method for identifying head shaking gestures and nodding gestures using an acceleration sensor device of a head-mounted device, wherein the acceleration sensor device provides acceleration signals for three linearly independent spatial directions, the method comprising the following steps: calculating the norm of the total acceleration based on the acceleration signal provided by the acceleration sensor device; and identifying head shaking gestures and nodding gestures using the calculated total acceleration norm; wherein the identification of the head shaking gesture is performed based on a comparison of the calculated total acceleration norm with a predetermined threshold value; and wherein the identification of the nodding gesture is performed based on the oscillation of the calculated total acceleration norm around a predetermined acceleration value.

Description

Technical Field

[0001] The present invention relates to a method for recognizing head gestures, namely head shaking gestures and nodding gestures, and a head-mounted device, in particular earplugs, headphones, smart glasses, etc. Background Art

[0002] Headphones with additional functionality are called "hearables." Users can interact or communicate with these devices through touch, gestures, or voice control. For example, a nod or shake of the head can be detected to convey a positive or negative message to the device. By using these gestures, users can control the settings and functions of the hearables with simple head movements.

[0003] Solutions for combining data from multiple sensors to track angular changes in the head in two or three dimensions, i.e., rotations around the x, y, and z axes, are known from US 2009 / 097689 A, TWM 482914U, US 2013 / 316679 A, ​​US 2015 / 003651 A, and US 2015 / 036835 A. These sensors may include acceleration sensors, rotational speed sensors, and pressure sensors.

[0004] Furthermore, solutions are known that evaluate only the measurement data of a single sensor: US 2012 / 020502 A uses data from a rotational speed sensor, CN 107277661 A uses data from a pressure sensor, and US 2006 / 140422 A uses data from an acceleration sensor.

[0005] Inertial sensors, such as acceleration sensors and rate sensors, are significantly more sensitive to changes in motion and more robust to environmental influences than other sensors (eg pressure sensors), and are therefore optimally suited for analyzing motion.

[0006] The change in the device's spatial orientation—that is, the change in rotation angle around the x-, y-, and z-axes—can be determined by fusing data from a rotational speed sensor with data from an acceleration sensor. The estimated orientation has high short-term accuracy, making it ideal for detecting short-term gestures. However, the rotational speed sensor, in particular, consumes relatively high power. Fusion of rotational speed sensor data with acceleration sensor data also results in high power consumption.

[0007] High accuracy in head gesture recognition can also be achieved by using only velocity sensor data, as nodding and shaking the head correspond to angular changes in different directions, which can be determined by integrating the velocity signal. However, the use of the velocity sensor also results in significantly higher power consumption than a solution based solely on acceleration sensor data.

[0008] If the device's coordinate system is aligned with the head's coordinate system, such as in smart glasses, a three-axis accelerometer is sufficient to determine the head's tilt angle. In this case, head tilt can be detected based on changes in the tilt angle. However, in headphones, and particularly earbuds, the device's coordinate system can deviate significantly from the head's coordinate system, depending on the user and their habits. Therefore, the device's tilt angle is generally insufficient to detect a nod. Furthermore, detecting a head shake based on the tilt angle is not possible. Summary of the Invention

[0009] The invention provides a method for detecting head shaking gestures and head nodding gestures and a device for a head mounted device having the features of the independent patent claims.

[0010] Preferred embodiments are the subject matter of the respective dependent claims.

[0011] According to a first aspect, the present invention therefore relates to a method for identifying head shaking gestures and nodding gestures using an acceleration sensor device of a head-mounted device, wherein the acceleration sensor device provides acceleration signals for three linearly independent spatial directions. A total acceleration norm is measured based on the acceleration signals provided by the acceleration sensor device. Head shaking gestures and nodding gestures are identified using the calculated total acceleration norm. Head shaking gestures are identified based on a comparison of the calculated total acceleration norm with a predetermined threshold value. Nodding gestures are identified based on the identification of oscillations of the calculated total acceleration norm around a predetermined acceleration value.

[0012] According to a second aspect, the present invention therefore relates to a device for a head-mounted device, which has an acceleration sensor device and a signal processing device. The acceleration sensor device provides acceleration signals for three linearly independent spatial directions. The signal processing device calculates a total acceleration norm based on the acceleration signals provided by the acceleration sensor device. The signal processing device identifies a head shaking gesture and a nodding gesture using the calculated total acceleration norm. The signal processing device performs the recognition of a head shaking gesture based on a comparison of the calculated total acceleration norm with a predetermined threshold value. The signal processing device performs the recognition of a nodding gesture based on the recognition of oscillations of the calculated total acceleration norm around a predetermined acceleration value.

[0013] Advantages of the present invention

[0014] The present invention provides a method for identifying head posture based solely on data from an acceleration sensor. This method is applicable not only to devices worn in the ear (e.g., earbuds) or on the ear (e.g., headphones), but also to eyewear. The method is robust with respect to the exact position in which the device is worn and is virtually unaffected by the sensor's orientation during wear. The method can achieve very high detection accuracy regardless of how the user wears the device. This results in a wide range of applications for different terminal devices, such as earbuds, headphones, and glasses.

[0015] The method is based solely on acceleration sensor data, significantly reducing power consumption compared to methods using a rotational speed sensor and enabling the recognition of both nodding and shaking heads. The method is therefore characterized by reduced power and resource consumption.

[0016] The principle underlying this method is explained in more detail below: If the acceleration sensor arrangement is at rest, the magnitude of the acceleration signal corresponds to the magnitude of the acceleration due to gravity, ie approximately 9.81 m / s 2 .

[0017] The acceleration that the acceleration sensor device is additionally subjected to due to the head movement when shaking the head is in the horizontal plane, ie perpendicular to the acceleration due to gravity. In this case, the magnitude of the acceleration signal is greater than the magnitude of the acceleration due to gravity.

[0018] When nodding, the acceleration generated by the head movement is in the same plane as the acceleration due to gravity. Depending on whether the head is moving upward or downward, the direction of the acceleration is either aligned with or opposite to the direction of the acceleration due to gravity. This means that the acceleration measured by the acceleration sensor device oscillates around the acceleration due to gravity. Distinguishing between shaking and nodding is made possible by the fact that the measured acceleration is greater than the acceleration due to gravity in the case of shaking the head, while oscillating around the acceleration due to gravity in the case of nodding.

[0019] The expression "total acceleration norm" is to be understood as a mathematical norm. Preferably, the Euclidean norm or the 2-norm can be used. However, other norms such as the sum norm, the 1-norm, or the p-norm can also be calculated.

[0020] According to an extension of the method for identifying head shaking and nodding gestures, a head shaking or nodding gesture is only identified when the time variation (e.g., time derivative) (i.e., jerk) of an acceleration signal provided by an acceleration sensor device, calculated based on the acceleration signal provided by the acceleration sensor device, exceeds a predetermined threshold value. This allows head gestures to be distinguished from other user movements. This is based on the consideration that the acceleration variation in the case of head shaking and nodding is greater than in normal use.

[0021] According to an extension of the method for detecting head shaking and nodding gestures, in order to detect whether the temporal variation of the acceleration signal exceeds a predetermined threshold value, a norm of the temporal variation of the acceleration signal is calculated from the acceleration signal provided by the acceleration sensor device. The norm of the temporal variation of the acceleration signal is filtered using a low-pass filter, wherein the filtered norm of the temporal variation of the acceleration signal is compared with the predetermined threshold value. The use of a low-pass filter can suppress high-frequency sensor noise.

[0022] According to one embodiment of the method for detecting head shaking and nodding gestures, in order to calculate the temporal profile of the acceleration signal, the magnitudes of the temporal derivatives of the acceleration signal are calculated and added together.

[0023] According to an extension of the method for identifying head shaking and nodding gestures, the calculated total acceleration norm is filtered using a low-pass filter, wherein head shaking and nodding gestures are identified based on the filtered total acceleration norm. The use of a low-pass filter can suppress high-frequency sensor noise.

[0024] According to an extension of this method for identifying head shaking and nodding gestures, the calculated total acceleration norm is filtered using a moving average filter, wherein head shaking and nodding gestures are identified based on the filtered total acceleration norm. The use of a moving average filter can save computing resources.

[0025] According to an extension of the method for identifying head shaking and nodding gestures, the calculated total acceleration norm is filtered using a filter with a reduced sampling rate, wherein head shaking and nodding gestures are identified based on the filtered total acceleration norm. Using a filter with a reduced sampling rate can save computing resources.

[0026] According to an embodiment of the method for detecting head shaking and nodding gestures, a head shaking gesture is detected only if the total acceleration norm exceeds a predetermined threshold value for at least one predetermined time period, thereby making it possible to distinguish head shaking gestures from other jerky events.

[0027] According to a development of the method for detecting a shaking head gesture and a nodding head gesture, the predetermined acceleration value for detecting a nodding head gesture corresponds to the acceleration due to gravity. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings show:

[0029] Figure 1: A schematic block diagram of a head mounted device according to one embodiment of the present invention;

[0030] Figure 2 : Schematic diagram of the time variation of the total acceleration norm; and

[0031] Figure 3 : A flowchart of a method for recognizing head shaking gestures and nodding gestures according to an embodiment of the present invention.

[0032] The numbering of the method steps is for the sake of clarity and should not generally imply a specific time sequence. In particular, several method steps can also be performed simultaneously. DETAILED DESCRIPTION

[0033] Figure 1 A schematic block diagram shows a device 1 for a head-mounted device, in particular for earbuds, headphones, smart glasses, etc. Device 1 includes an acceleration sensor arrangement 2 having three sensor components 21, 22, 23, which each detect acceleration along a spatial direction, wherein these spatial directions are linearly independent. In particular, these spatial directions can be perpendicular to one another.

[0034] The acceleration signals generated by the sensor components 21, 22, and 23 are output to the signal processing device 3 of the device 1 and processed by it. The signal processing device 3 first distinguishes the head posture from other events. To this end, the time variation (time derivative) of the acceleration signal is calculated. In addition, the norm of the time variation of the acceleration signal is calculated and then optionally filtered with the aid of a low-pass filter. The filtered norm of the time variation of the acceleration signal is compared with a predetermined threshold value. If the threshold value is exceeded, a head posture is detected.

[0035] In parallel or subsequently, the signal processing device 3 checks whether a shaking head gesture or a nodding gesture is present. To this end, the signal processing device 3 calculates a total acceleration norm based on the acceleration signal provided by the acceleration sensor device 2. Optionally, the calculated norm is then filtered using a low-pass filter, a moving average filter, and / or a filter with a reduced sampling rate. The calculated total acceleration norm is compared with a predetermined threshold value. If the threshold value is exceeded within a predetermined time period, the signal processing device 3 recognizes a shaking head gesture.

[0036] If the threshold value is not exceeded, the signal processing device 3 checks whether the calculated total acceleration norm oscillates around a predetermined acceleration value. For example, the signal processing device 3 can check whether the predetermined acceleration value is exceeded or undershot more frequently than a predetermined number of times within a predetermined time window. If oscillations are detected, the signal processing device 3 recognizes a nodding gesture.

[0037] Figure 2 A schematic diagram shows the time course of the total acceleration norm for alternating head shaking and nodding. The first signal curve B1 corresponds to a head shaking gesture. Here, a higher value of the calculated total acceleration norm A occurs, meaning a predetermined threshold value is exceeded. The second signal curve B2 corresponds to a nodding gesture. Here, the calculated total acceleration norm A oscillates around the gravitational acceleration g.

[0038] Figure 3 A flow chart of a method for recognizing head shaking gestures and head nodding gestures is shown.

[0039] In a first step S1 , the sensor components 21 , 22 , 23 of the acceleration sensor arrangement 2 of the headset generate acceleration signals for three linearly independent spatial directions.

[0040] It is detected at least partially in parallel or also in temporally successive order whether a head gesture is present S2 and, if a head gesture is present, whether a head shaking gesture or a head nodding gesture is present S3.

[0041] To identify whether a head gesture exists, S2, the time derivative of the acceleration signal is first calculated, S21. The norm of the time variation of the acceleration signal, such as the L1 norm, is further calculated. To this end, the time variation of each acceleration signal is calculated separately and added, S22.

[0042] The norm of the time variation of the acceleration signal is filtered using a low-pass filter (S23), and the filtered norm of the time variation of the acceleration signal is compared with a predetermined threshold (S24). If the threshold is exceeded, a head gesture is detected (S25). However, it is not yet known whether the head gesture is a head shake or a nod. This is determined in step S3, where features are extracted based on which the detected head gesture can be classified as a head shake or a nod.

[0043] In order to identify whether the head gesture is a shaking head gesture or a nodding head gesture, S3, first, a total acceleration norm is calculated, S31, based on the acceleration signal provided by the acceleration sensor device 2. In a stable state, this norm is equivalent to the magnitude of the gravitational acceleration.

[0044] The calculated total acceleration norm is filtered by means of a low-pass filter, S32. The filtered total acceleration norm is further additionally filtered by means of a moving average filter for determining a smoothed average value or by means of a filter with a reduced sampling rate, S33.

[0045] The filtered total acceleration norm is compared with a predetermined threshold value. If the threshold value is exceeded within a predetermined time period, a head shaking gesture is detected as long as the head gesture is present, S34. If the actual presence of the head gesture is detected in step S25, a head shaking gesture is detected, S41.

[0046] If it is detected that the filtered total acceleration norm oscillates around the magnitude of the gravitational acceleration within a predetermined time period, a nodding gesture is detected as long as a head gesture is present. If it is detected in step S25 that a head gesture is actually present, a nodding gesture is detected, S42.

Claims

1. A method for detecting shaking and nodding gestures using an acceleration sensor device (2) of a head-mounted device, wherein: The acceleration sensor device (2) provides acceleration signals for three linearly independent spatial directions. The method comprises the following steps: Calculating (S31) a total acceleration norm based on the acceleration signal provided by the acceleration sensor device (2); and identifying (S4) a head shaking gesture and a head nodding gesture using the calculated total acceleration norm; wherein the head shaking gesture is identified based on a comparison between the calculated total acceleration norm and a predetermined threshold value (S42); and The recognition of the nodding gesture is performed based on the recognition of oscillations of the calculated total acceleration norm around a predetermined acceleration value ( S41 ).

2. The method according to claim 1, wherein The shaking or nodding gesture is detected only when a temporal variation of the acceleration signal calculated from the acceleration signal provided by the acceleration sensor device (2) exceeds (S2) a predetermined threshold value.

3. The method according to claim 2, wherein: In order to identify whether the time variation of the acceleration signal exceeds a predetermined threshold value, the norm of the time variation of the acceleration signal is calculated (S22) based on the acceleration signal provided by the acceleration sensor device (2), wherein the norm of the time variation of the acceleration signal is filtered by means of a low-pass filter (S23), and wherein the filtered norm of the time variation of the acceleration signal is compared with the predetermined threshold value (S24).

4. The method according to claim 3, wherein: In order to calculate ( S22 ) the norm of the time variation of the acceleration signal, the magnitudes of the time derivatives of the acceleration signal are calculated and the magnitudes are added.

5. A method according to any one of the preceding claims, wherein The calculated total acceleration norm is filtered by means of a low-pass filter ( S32 ), and the head shaking gesture and the nodding gesture are identified based on the filtered total acceleration norm.

6. A method according to any one of the preceding claims, wherein The calculated total acceleration norm is filtered by means of a moving average filter (S33), and the head shaking gesture and the nodding gesture are identified based on the filtered total acceleration norm.

7. A method according to any one of the preceding claims, wherein The calculated total acceleration norm is filtered by means of a filter with a reduced sampling rate ( S33 ), and a head shaking gesture and a head nodding gesture are identified based on the filtered total acceleration norm.

8. A method according to any one of the preceding claims, wherein A head shaking gesture is detected only if the total acceleration norm exceeds the predefined threshold value for at least one predefined time period.

9. A method according to any one of the preceding claims, wherein The predetermined acceleration value for detecting a nodding gesture corresponds to the acceleration due to gravity.

10. A device for a head-mounted appliance (1), the device comprising: an acceleration sensor device (2) configured to provide acceleration signals for three linearly independent spatial directions; and A signal processing device (3), the signal processing device being configured to: a. calculating the total acceleration norm based on the acceleration signal provided by the acceleration sensor device (2), and b. Identify head shaking gestures and nodding gestures using the calculated total acceleration norm, in, The signal processing device (3) is configured to perform the recognition of the head shaking gesture based on a comparison of the calculated total acceleration norm with a predetermined threshold value; and The signal processing device (3) is designed to perform the recognition of the nodding gesture based on the recognition of oscillations of the calculated total acceleration norm around a predetermined acceleration value.

Images