A smart watch gesture interaction control method and system

Abstract

The application discloses a smart watch gesture interaction control method and system, aiming at solving the technical problems of low reference drift compensation precision, high calculation complexity, rising power consumption and high real-time recognition delay in the prior art. The method comprises the following steps: collecting original data of an inertial sensor and performing mean filtering pretreatment; using an adaptive forgetting factor recursive least square method to estimate and update a reference point in real time; realizing dynamic gesture segmentation based on acceleration amplitude double threshold detection; extracting the main motion direction, motion amplitude and statistical characteristics of the angular velocity signal to form a multi-dimensional feature vector; performing lightweight gesture recognition; and mapping the recognition result into a control instruction and executing it. Through the above technical solution, the calculation complexity is reduced from O(N 2 ) to O(NW), the recognition delay is controlled within a preset threshold, the average power consumption during continuous operation does not exceed a preset power consumption threshold, the cumulative error problem is effectively overcome, and high-accuracy, low-delay and low-power real-time gesture interaction control is realized.

Description

Technical Field

[0001] This invention belongs to the field of smartwatch control technology, specifically, it relates to a smartwatch gesture interaction control method and system. Background Technology

[0002] With the rapid development of mobile internet and IoT technologies, smart wearable devices have gradually become an important terminal form in the field of human-computer interaction. Smartwatches, as a typical representative of wearable devices, integrate sensor technology, low-power computing technology, and wireless communication technology, enabling diverse functions such as health monitoring, message notifications, and mobile payments. Currently, human-computer interaction is undergoing a profound transformation from traditional touchscreen operation to natural interaction modes. Among these, gesture interaction, as an interaction paradigm that conforms to human intuition, has become a key technological direction for improving the user experience of smart devices.

[0003] Gesture interaction control aims to achieve contactless control of smart devices by recognizing the user's hand movement trajectory and posture changes. This method collects raw data from inertial sensors such as accelerometers and gyroscopes, and through signal processing, feature extraction, and pattern classification, maps physical gestures into corresponding device control commands. In the context of smartwatches, where screen size is limited, wearing position is fixed, and users are often in motion, gesture interaction can effectively compensate for the shortcomings of touch operation, expanding the dimensions of interaction and operational convenience.

[0004] Existing gesture interaction methods for smartwatches generally suffer from the following problems: Traditional threshold-matching methods rely on predefined fixed parameters, resulting in poor adaptability to differences in the physiological characteristics of different users and a tendency to decrease recognition accuracy; while statistical learning-based methods can improve generalization ability, their high computational complexity leads to a sharp increase in power consumption on resource-constrained devices such as smartwatches, severely impacting battery life; furthermore, existing dynamic time warping algorithms are computationally intensive and have high latency when processing real-time continuous gesture streams, making it difficult to meet the immediacy requirements of gesture interaction; and thirdly, due to the loosening or shifting of the wearing position of smartwatches, the reference point of sensor data drifts over time, causing cumulative errors to gradually increase and interaction reliability to continuously decrease. Therefore, how to achieve high-precision, low-power, and robust real-time gesture interaction control on resource-constrained smartwatches has become an urgent technical challenge to be solved in this field. Summary of the Invention

[0005] The purpose of this invention is to provide a gesture interaction control method and system for smartwatches, mainly to solve the technical problems in the prior art such as low reference drift compensation accuracy, high computational complexity leading to increased power consumption, and high real-time recognition latency.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A gesture interaction control method for a smartwatch includes the following steps:

[0008] S1 collects three-dimensional acceleration and three-dimensional angular velocity signals during the user's gesture movement. Each frame of data includes a timestamp, three-axis acceleration values, and three-axis angular velocity values.

[0009] S2 performs mean filtering on the collected triaxial acceleration and triaxial angular velocity signals respectively, and converts the data in the local coordinate system of the sensor to the coordinate system of the smartwatch body;

[0010] S3 estimates and updates the reference point for gesture movement in real time, and uses the adaptive forgetting factor recursive least squares method to estimate the reference drift in order to overcome the cumulative error problem caused by the loosening and shifting of the smartwatch wearing position.

[0011] S4, based on the compensated acceleration signal amplitude, performs real-time monitoring, and determines the start and end of the gesture by the acceleration amplitude of multiple consecutive sampling points, thereby realizing real-time segmentation of the continuous gesture stream;

[0012] S5, perform piecewise integration on the angular velocity signal within the gesture segment, calculate the main motion direction and motion amplitude of the gesture, and extract the gesture features of the angular velocity signal to form a multi-dimensional feature vector;

[0013] S6. Based on the multidimensional feature vector, an optimized dynamic time warping algorithm is used to match and recognize the gesture template library.

[0014] S7 maps the corresponding gesture to predefined smartwatch control commands based on the recognition results, and executes the corresponding function operations through the smartwatch's processor.

[0015] Furthermore, in step S1, the three-dimensional acceleration signal and the three-dimensional angular velocity signal are synchronously acquired by the accelerometer and gyroscope built into the smartwatch, and a synchronous trigger signal is generated in each sampling cycle by a unified hardware timer to ensure that the acceleration signal and the angular velocity signal are strictly aligned in the time dimension.

[0016] Furthermore, in S2, the mean filtering process adopts a sliding window mechanism, and the filter window length is determined based on a trade-off analysis of computational complexity and filtering effect; the coordinate system transformation adopts a static calibration method, and the axial deviation angle between the sensor local coordinate system and the smartwatch body coordinate system is obtained through factory calibration, and compensation is performed during the data processing stage to ensure the consistency of the two coordinate systems.

[0017] Furthermore, in S3, the forgetting factor of the adaptive forgetting factor recursive least squares method is set based on the balance between the tracking speed of benchmark changes and the noise suppression capability; the benchmark update cycle eliminates the accumulated error over a long period of use by periodically updating the benchmark reference point, and the compensated benchmark reference point is used to normalize the subsequent feature extraction process.

[0018] Furthermore, in S4, the specific method for determining the start and end of a gesture is as follows: when the acceleration amplitude of multiple consecutive sampling points exceeds a preset dynamic threshold, it is determined that the gesture has started; when the acceleration amplitude of multiple consecutive sampling points is lower than the dynamic threshold, it is determined that the gesture has ended. The dynamic threshold is set based on statistical analysis of the distribution of acceleration amplitude in static and dynamic states.

[0019] Furthermore, in S5, the segmented integration of the angular velocity signal adopts the trapezoidal integration method, which divides the duration of the gesture segment into multiple equally spaced time windows, and performs numerical integration of the angular velocity signal within each time window to obtain the angle increment.

[0020] Furthermore, the multidimensional feature vector extracts the number of peaks, peak intervals, and signal energy distribution of the angular velocity signal simultaneously as statistical features.

[0021] Furthermore, the dynamic time warping algorithm employs Sakoe-Chiba constraints to limit the search range of the matching path, and adopts a hierarchical recognition strategy. First, it performs broad category filtering based on the number of peaks, and then performs fine matching within each category. The search range is limited to each preset bandwidth sampling point on both sides of the diagonal. The bandwidth parameter is set according to the balance between recognition accuracy and computational efficiency. The larger the bandwidth, the wider the search range but the greater the computational cost; the smaller the bandwidth, the less computation is required. The hierarchical recognition strategy uses the number of peaks for pre-classification, which greatly compresses the number of candidate templates, and then performs fine dynamic time warping matching on this basis.

[0022] Furthermore, the gesture template library contains a variety of predefined gesture templates, each gesture template contains multiple reference samples to cover the differences in different execution methods, and the total storage capacity of the template library is controlled within a preset storage capacity; it supports users to add new gesture templates, and the custom templates are generated and added to the template library through the same feature extraction process.

[0023] Based on the above method, the present invention also provides a smartwatch gesture interaction control system, comprising:

[0024] The data acquisition module is used to simultaneously acquire three-dimensional acceleration signals and three-dimensional angular velocity signals during the user's gesture movements;

[0025] The preprocessing module is used to perform mean filtering and coordinate transformation on the three-dimensional acceleration signal and the three-dimensional angular velocity signal, respectively.

[0026] The reference drift compensation module is used to estimate and update the reference point of gesture movement in real time to overcome the cumulative error caused by the loosening and shifting of the smartwatch wearing position;

[0027] The dynamic gesture segmentation module is used for real-time monitoring based on the compensated acceleration signal amplitude. It determines the start and end of a gesture by comparing the acceleration amplitude of multiple consecutive sampling points with a dynamic threshold, thereby achieving real-time segmentation of a continuous gesture stream.

[0028] The feature extraction module is used to calculate the main motion direction and motion amplitude of the gesture, and at the same time extract the number of peaks, peak interval and signal energy distribution of the angular velocity signal as statistical features to form a multi-dimensional feature vector;

[0029] The gesture recognition module is used to perform matching and recognition based on the multi-dimensional feature vector, and combines a hierarchical recognition strategy to first filter the categories based on the number of peaks and then perform fine matching within the categories to output the corresponding gesture category.

[0030] The control command generation module is used to find the corresponding control command code based on the recognized gesture category and generate the corresponding control command.

[0031] The execution module is used to execute the functional operations corresponding to the control commands through the processor of the smartwatch.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] (1) This invention employs an adaptive forgetting factor recursive least squares method to estimate and update the reference point for gesture movement in real time. Through an adaptive adjustment strategy for the forgetting factor—automatically reducing the forgetting factor to accelerate tracking when the prediction error exceeds a preset threshold, and appropriately increasing the forgetting factor to enhance estimation smoothness when the prediction error is below the threshold—online compensation is achieved for sensor reference drift caused by loosening of the smartwatch's wearing position. Compared to existing static calibration methods that rely on fixed parameters, this invention can dynamically track reference changes caused by factors such as wrist skin elastic deformation and strap tension relaxation, effectively eliminating reference deviations accumulated during long-term use. This ensures that feature extraction is always based on accurate normalized reference points, thereby significantly improving the reliability and stability of gesture recognition in long-term continuous use scenarios.

[0034] (2) This invention introduces Sakoe-Chiba band constraints into the dynamic time warping algorithm, limiting the search range of the matching path to within the preset bandwidth sampling points on both sides of the diagonal, thus reducing the time complexity of the algorithm from O(N) of the traditional method.2 ) decreased to O(N (W); Simultaneously, a hierarchical recognition strategy is adopted. First, broad category filtering is performed based on the peak number of angular velocity signals, significantly compressing the number of candidate templates. Then, fine dynamic time warping matching is performed within each category. Through the above two-level optimization mechanism, this invention significantly reduces computational overhead while ensuring recognition accuracy. This allows the gesture recognition algorithm to run smoothly on embedded platforms such as smartwatches where processor computing power and battery capacity are limited, effectively reducing the average power consumption during continuous operation and extending the device's battery life.

[0035] (3) Based on the compensated acceleration signal amplitude, this invention employs a dual-threshold and continuity detection strategy—determining the start of a gesture by multiple consecutive sampling points exceeding a dynamic threshold and determining the end of a gesture by multiple consecutive sampling points falling below the dynamic threshold—achieving real-time segmentation of continuous gesture streams. This method, combined with a state machine mechanism (idle state, pre-detection state, gesture in progress state, gesture end state), can accurately distinguish between the user's conscious gesture actions and unintentional movements in daily activities, effectively avoiding false triggering. Simultaneously, gesture segmentation, feature extraction, and recognition matching form a pipelined processing architecture, controlling the recognition delay within a preset threshold, making the user almost imperceptible to noticeable response delays, significantly improving the naturalness and immediacy of gesture interaction, and enhancing the user experience of contactless operation on smartwatches. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of the overall technical architecture of the control system of the present invention;

[0037] Figure 2 This is a flowchart illustrating the logical process of multidimensional feature vector extraction in this invention.

[0038] Figure 3 This is a schematic diagram of the core principle framework of gesture recognition in this invention; Detailed Implementation

[0039] The present invention will be further described below with reference to the accompanying drawings and embodiments. The embodiments of the present invention include, but are not limited to, the following embodiments.

[0040] Example 1

[0041] like Figure 1As shown, this invention discloses a gesture interaction control method and system for smartwatches. The system mainly consists of eight modules: data acquisition, data preprocessing, reference drift compensation, dynamic gesture segmentation, feature extraction, gesture recognition, and control command generation and execution. The system relies on inertial sensors to synchronously acquire raw data, sequentially completing processes such as data preprocessing, reference drift compensation based on adaptive forgetting factor recursive least squares method, dynamic gesture segmentation, multi-dimensional feature vector extraction, lightweight dynamic time warping gesture recognition, and control command mapping. Ultimately, it achieves high-precision, low-power, and customizable gesture interaction control for smartwatches.

[0042] First, the smartwatch's built-in six-axis inertial measurement unit includes a three-axis accelerometer and a three-axis gyroscope. The accelerometer collects the linear acceleration components generated by the user's wrist movements, while the gyroscope collects the angular velocity components generated by wrist rotation. During data acquisition, the accelerometer and gyroscope employ a synchronous trigger acquisition mechanism. A unified hardware timer generates a synchronous trigger signal in each sampling cycle, ensuring strict alignment of the two sets of sensor data in the time dimension. The accelerometer's sampling frequency is set to a preset frequency, and the gyroscope's sampling frequency is also set to a preset frequency, maintaining consistency between the two sampling rates for subsequent feature fusion processing. Each frame of sensor data includes a precise timestamp field, three-axis acceleration values ​​(corresponding to the acceleration components in the X, Y, and Z axes, in meters per second squared) and three-axis angular velocity values ​​(corresponding to the angular velocity components in the X, Y, and Z axes, in radians per second). The timestamp is obtained using a high-precision timer from the system clock, with a time resolution better than microseconds, ensuring the accuracy and uniqueness of the timestamp. In practical configurations, the typical preset sampling frequency is 100Hz, a value that comprehensively considers the frequency characteristics of human hand gestures. The frequency range of hand gestures is typically concentrated in the low-to-mid-high frequency range of 0.5Hz to 5Hz. A sampling frequency of 100Hz can capture the detailed changes during hand gestures with sufficient resolution, while avoiding data redundancy and increased power consumption caused by excessively high sampling rates. The sampling frequency setting also takes into account the constraints of computing resources and power budget, achieving optimal data acquisition quality under limited processor computing power and battery capacity. The data acquisition module continuously runs in a background thread, buffering the acquired raw data frames through a circular buffer and triggering semaphores for subsequent preprocessing processes to ensure the continuity and real-time nature of the data stream.

[0043] Next, the acquired raw acceleration and angular velocity signals are subjected to mean filtering to eliminate high-frequency noise and transient fluctuation interference. The mean filter employs a sliding window mechanism, with the filter window length set to a preset window length. In practice, a typical preset window length is 5 sampling points, corresponding to a 50-millisecond time window. This parameter value is based on a trade-off analysis between computational complexity and filtering effect: a shorter window length can retain useful high-frequency motion information, which is crucial for capturing the details of rapid gestures, while effectively suppressing high-frequency noise interference and avoiding false triggers and feature deviations caused by noise. The specific operation of mean filtering is as follows: for the current sampling time... Filter output It equals the arithmetic mean of all sampled points within the window, i.e. Where N is the window length. The filtered signal output is sent to the subsequent coordinate transformation module. The coordinate transformation adopts a static calibration method. During the smartwatch manufacturing process, the axial deviation angle between the sensor's local coordinate system and the smartwatch's body coordinate system is obtained through professional calibration equipment. The calibration process is carried out in a normal temperature environment, and random errors are eliminated by averaging multiple measurements in multiple postures to obtain accurate rotation matrix parameters. The specific value of the preset window length is determined based on the sensor's noise characteristics. For the typical six-axis inertial sensor used in this invention, a window length of 5 can effectively filter out noise frequency components higher than 20Hz, while controlling the amplitude attenuation of useful signals (below 5Hz) within an acceptable range. In the data processing stage, the system reads the pre-stored calibration parameters and transforms the three-axis acceleration and three-axis angular velocity data in the sensor's local coordinate system to the smartwatch's body coordinate system through coordinate rotation transformation. The mathematical expression of the coordinate transformation is:

[0044]

[0045] in, A vector in the fuselage coordinate system. Let R be a vector in the sensor's local coordinate system, and let R be a 3×3 rotation matrix obtained through calibration. The elements of the rotation matrix R are obtained as follows: First, the smartwatch is fixed on a standard attitude calibration frame, and each axis is sequentially aligned with the direction of gravity, recording the triaxial acceleration readings as a reference. Then, continuous rotation is performed in measurement mode, and the optimal estimate of the axial deviation angle is obtained using the least squares method. Coordinate transformation ensures the consistency between the sensor data and the smartwatch's geometry, providing a unified data reference for subsequent baseline drift compensation and feature extraction.

[0046] Then, based on the preprocessed sensor data, the reference point for gesture movement is estimated and updated in real time through online learning. An adaptive forgetting factor recursive least squares method is used to estimate the reference drift, overcoming the cumulative error caused by the loosening of the smartwatch's wearing position. In actual smartwatch usage scenarios, due to factors such as elastic deformation of the wrist skin, changes in the coefficient of friction due to sweating, and natural relaxation of the strap tension, the sensor's installation position relative to the wrist will experience slight shifts and tilts over time. This drift manifests as a gradual increase in the rotation angle of the sensor coordinate system relative to the device's coordinate system, leading to systematic deviations in the collected acceleration and angular velocity data. Gesture feature extraction and recognition rely on accurate reference points; reference drift directly affects the accuracy of feature calculations, thus reducing recognition accuracy. The basic principle of the recursive least squares method is to fit the observed data by continuously updating the parameter estimates. Let the parameter vector of the reference point to be estimated be... The observation vector is Then the recursive update formula for parameter estimation is:

[0047]

[0048]

[0049] in, Let covariance matrix be the variance matrix. Forgetting factor, This represents the current observation value. In this embodiment, the forgetting factor is set to 0.95. This value is based on a balance between the tracking speed of benchmark changes and the noise suppression capability: the larger the forgetting factor, the slower the decay of historical data, and the smoother the estimation result, but the slower the response speed to benchmark abrupt changes; the smaller the forgetting factor, the more sensitive the tracking of benchmark abrupt changes, but the estimation result is easily affected by noise interference. The benchmark update cycle is set to a preset update cycle, typically performed every 200 milliseconds for online updating of benchmark parameters. Periodically updating the benchmark reference point eliminates errors accumulated over long periods of use, and the compensated benchmark reference point is used to normalize subsequent feature extraction processes. Within each update cycle, the system extracts statistical features from the preprocessed sensor data of the most recent cycle, including the mean and variance of accelerations along each axis and the mean of angular velocities, as input observation vectors for the recursive least squares method.

[0050] The adaptive adjustment strategy for the forgetting factor is based on real-time monitoring of prediction errors: when the prediction error exceeds a preset error threshold, the forgetting factor is automatically reduced to accelerate tracking; when the prediction error is below the preset threshold and persists for several cycles, the forgetting factor is appropriately increased to enhance the smoothness of the estimation results. The compensated benchmark reference point is stored in quaternion form, which offers advantages in numerical stability and computational efficiency compared to Euler angle representation. The system maintains a sliding estimation window for a benchmark reference point, with the window length set as a reasonable multiple of the benchmark update cycle to ensure robustness and noise resistance in benchmark drift estimation.

[0051] Real-time monitoring is performed based on the compensated acceleration signal amplitude. When the acceleration amplitude at multiple consecutive sampling points exceeds a preset dynamic threshold, the gesture is considered to have started; when the acceleration amplitude at multiple consecutive sampling points falls below the dynamic threshold, the gesture is considered to have ended, thus achieving real-time segmentation of the continuous gesture stream. The dynamic gesture segmentation module continuously analyzes the temporal variation characteristics of the acceleration signal amplitude. The amplitude calculation formula is as follows:

[0052]

[0053] in , , These represent the acceleration components along the X, Y, and Z axes, respectively. The dynamic threshold is set based on statistical analysis of the acceleration amplitude distribution in both static and dynamic states. In the static state, the acceleration amplitude mainly includes the gravitational acceleration component (approximately 9.8 m / s²). 2 The sensor's zero-point offset and noise components fluctuate around the gravitational acceleration value with relatively small amplitude. During hand gestures, the acceleration amplitude increases or changes significantly due to the active movement and rotation of the wrist. A typical preset dynamic threshold is 1.1 times the gravitational acceleration value, approximately 10.78 m / s². 2 This threshold effectively distinguishes between static and dynamic states, while avoiding false triggers caused by minor wrist tremors or weak movements due to breathing. Dynamic gesture segmentation employs a strategy of dual thresholds plus continuity detection.

[0054] The start of a gesture is determined by multiple consecutive sampling points exceeding a dynamic threshold to avoid false triggering due to noise. The number of sampling points for continuity detection is set to three consecutive sampling points; that is, a gesture is considered to have started only when the acceleration amplitude exceeds the dynamic threshold for three consecutive sampling periods. The end of a gesture is determined by multiple consecutive sampling points falling below the dynamic threshold to distinguish between a pause and a true end to the gesture. The end of a gesture also uses continuity detection, with five consecutive sampling points below the threshold, a more conservative approach than the start of a gesture to avoid premature truncation of the gesture action leading to incomplete features. The segmented gesture segments contain valid motion information for subsequent feature extraction and recognition. For each segmented gesture segment, the system records its start and end timestamps, as well as the complete raw data sequence of acceleration and angular velocity within the segment. The minimum length of a gesture segment is limited to at least 10 consecutive sampling points exceeding the threshold to ensure the extracted features are statistically significant. The maximum length of a gesture segment is limited to no more than 500 consecutive sampling points exceeding the threshold to avoid the computational burden caused by excessively long segments due to continuous user movement. When the system detects that the length of a gesture segment has reached the maximum length limit, it forcibly terminates the current segment and begins the next round of segmentation detection. During the processing of continuous gesture streams, the system maintains a state machine, which includes an idle state, a pre-detection state, a gesture in progress state, and a gesture termination state, and switches between these states based on changes in the acceleration amplitude.

[0055] Then, the angular velocity signal within the gesture segment is integrated piecewise to calculate the main motion direction and amplitude of the gesture. Simultaneously, the number of peaks, peak intervals, and signal energy distribution of the angular velocity signal are extracted as statistical features to form a multidimensional feature vector. For example... Figure 2As shown, the piecewise integration of the angular velocity signal employs the trapezoidal integration method. The duration of the gesture segment is divided into multiple equally spaced time windows. Within each time window, the angular velocity signal is numerically integrated to obtain the angle increment. The main motion direction is determined through principal component analysis of the integration results: for the three-dimensional angle increment sequence obtained by integration, its covariance matrix is ​​calculated and eigenvalue decomposition is performed. The eigenvector corresponding to the largest eigenvalue points to the main motion direction of the gesture. The main motion direction is quantized within a 360-degree range, with a quantization precision of 1 degree, meaning the main motion direction is represented by an integer ranging from 0 to 359. The motion amplitude is obtained by calculating the total modulus of the integrated angle increment. The motion amplitude is normalized within a preset motion amplitude range, typically from 0 to 100. Peak detection uses a smoothing difference algorithm. First, the original angular velocity signal is Gaussian smoothed to eliminate noise. Then, the first-order difference of the smoothed signal is calculated. The zero-crossing point where the difference sign changes from positive to negative corresponds to the local maximum point of the signal, i.e., the peak value. The Gaussian smoothing parameters of the smoothing difference algorithm are set to a smoothing window length of 5 sampling points and a standard deviation of 1.2. This parameter configuration can accurately locate peak positions while effectively suppressing noise. The peak interval is defined as the time difference between adjacent peaks, expressed in terms of the number of sampling points, and needs to be divided by the sampling frequency to convert it to actual time. The signal energy distribution is obtained by calculating the variance and skewness of the angular velocity signal: variance describes the energy concentration of the signal, and skewness describes the asymmetric distribution of signal energy on the time axis. The extracted feature vector has a preset dimension, typically configured as a 12-dimensional feature vector, including the number of peaks of the three-axis angular velocities (one dimension each), the mean of the peak intervals of the three-axis angular velocities (one dimension each), the variance of the three-axis angular velocity signal (one dimension each), the quantized value of the main motion direction (one dimension), and the normalized value of the motion amplitude (one dimension). During feature extraction, the system simultaneously calculates the statistical significance evaluation of each feature for confidence evaluation in the subsequent recognition process. For each segmented gesture segment, the system outputs the corresponding multi-dimensional feature vector and the original data pointer of the segment for use by the subsequent recognition module.

[0056] Next, as Figure 3 As shown, based on the multidimensional feature vector, an optimized dynamic time warping algorithm is used to match and recognize the target gesture template library. The dynamic time warping algorithm employs Sakoe-Chiba constraints, with the bandwidth set to a preset bandwidth parameter to limit the search range of the matching path, thus reducing the computational complexity from O(N) of traditional methods. 2 ) reduced to O(N W). The core objective of dynamic time warping algorithms is to calculate the optimal alignment path and its cumulative distance between two time series. In gesture recognition applications, one time series is the feature sequence of the gesture to be recognized, and the other time series is a reference template sequence in a gesture template library. Traditional dynamic time warping algorithms allow the matching path to select an alignment point at any position in the time dimension, and its cumulative distance is calculated using the following formula:

[0057]

[0058] in For the two sequences at position The local distance at a given point. When both sequences have length N, the search space for the optimal path is N. For a two-dimensional matrix of N, the time complexity of the algorithm is O(N). 2 The Sakoe-Chiba band constraint significantly reduces computational complexity by limiting the search range. Under this constraint, the matching path is restricted to sampling points with a preset bandwidth on both sides of the diagonal; that is, only when the conditions are met... nodes Only when the optimal path is calculated does it participate in the calculation, where W is the bandwidth parameter. A typical preset bandwidth parameter value is N / 4, which is one-quarter of the sequence length. The bandwidth parameter setting is based on a balance between recognition accuracy and computational efficiency: a smaller bandwidth can significantly reduce computation but may lose effective matching paths, leading to a decrease in recognition accuracy; a larger bandwidth is closer to the search capability of traditional algorithms but increases computational overhead. In this embodiment, the bandwidth parameter is dynamically adjusted according to the gesture segment length using an adaptive strategy: for short gestures with a length of less than 50 sampling points, a larger bandwidth is used to ensure matching accuracy; for long gestures with a length of more than 100 sampling points, a smaller bandwidth is used to control computation. After applying the Sakoe-Chiba band constraint, the algorithm's time complexity is reduced to O(N). W), where N is the sequence length and W is the bandwidth. In a typical configuration, N is 100 to 200 sampling points and W is 25 to 50 sampling points, reducing computational cost by approximately 4 to 8 times compared to traditional methods. A hierarchical recognition strategy further improves recognition efficiency. This strategy employs a two-tier architecture: first, it performs broad category filtering based on the number of peak values, and then performs fine-grained matching within each category. During the broad category filtering stage, the system counts the number of peak values ​​for the gesture to be recognized and restricts candidate templates to a subset of templates with the same range of peak values.

[0059] The gesture template library contains a variety of predefined gesture templates, each with multiple reference samples to cover differences in execution methods. The total storage capacity of the gesture template library is controlled within a preset storage capacity, typically 512 kilobytes, and template feature vectors are stored using an efficient data compression format. When a user adds a new gesture template, the feature vector for that gesture is generated through the same feature extraction process and added to the corresponding gesture type group in the template library. The recognition accuracy of user-defined gestures remains at a relatively high preset level, typically above 90%.

[0060] The weights of reference samples in the template library are dynamically adjusted based on the user's historical usage data to achieve a personalized adaptation mechanism. The weight adjustment uses an exponential weighting method. The weight of samples that have been successfully identified the most recently (with a preset number of successful identifications) is set as the first preset weight value, typically 0.8; the weight of historical samples is set as the second preset weight value, typically 0.2. The recursive formula for weight adjustment is: Where α is the forgetting coefficient and δ is the indicator of whether the current recognition was successful. A confidence assessment is introduced during the recognition phase. If the ratio of the best match score to the second-best match score is lower than a preset confidence multiple, the recognition is considered uncertain. A typical preset confidence multiple is 1.5. The system triggers a gesture confirmation prompt, requiring the user to re-execute the gesture or confirm the current recognition result to avoid accidental triggering and erroneous control.

[0061] Taking the wrist-flipping motion as an example, the peak number range is set to 2 to 4 peaks; the peak number range for the circling motion is set to 4 to 8 peaks. Through peak number filtering, the number of candidate templates is compressed from 100% of the total template library capacity to a preset ratio range, typically 10% to 20%. Based on this, the system performs fine-grained dynamic time warping matching within the filtered template subset, calculating the distance score between the gesture to be recognized and each candidate template. The recognition result is taken as the gesture type corresponding to the template with the smallest distance score as the final recognition output. The overall recognition accuracy reaches a preset high recognition accuracy level, typically above 95%; the recognition latency is controlled within a preset latency threshold, typically below 50 milliseconds, meeting the immediacy requirements of gesture interaction.

[0062] Finally, based on the recognition results, the corresponding gestures are mapped to predefined smartwatch control commands, and the smartwatch's processor executes the corresponding functional operations. Each predefined gesture in the gesture template library is associated with a control command code, which corresponds to the system interface of the smartwatch's operating system. When the gesture recognition module outputs the recognition result, the system looks up the control command code corresponding to that gesture type and triggers the corresponding functional operation through the system call interface. The mapping relationship between predefined gestures and control commands includes: wrist flipping is mapped to a screen scrolling command, enabling scrolling the current interface content up or down, with the scrolling direction determined by the main movement direction of the wrist flip; wrist raising is mapped to a screen-on command, triggering the screen-on operation when it is off; wrist shaking is mapped to an application switching command, ending the current application and returning to the main interface; tapping is mapped to a confirmation selection command, equivalent to a click operation; and drawing a circle is mapped to a volume adjustment command, with the direction of the circle determining whether the volume increases or decreases. Before executing each control command, the system performs a security check to ensure the legality of the command. Security checks include: verifying whether the current system status allows the execution of the command (e.g., denying network-related commands in airplane mode), checking user authorization settings (e.g., permission restrictions in parental control mode), and preventing command injection attacks (whitelisting the command parameters). After execution, the system records the operation log of this gesture interaction, including gesture type, timestamp, control command, and execution result, for use in subsequent personalized adaptation mechanism training data collection.

[0063] The following explanation uses a specific application scenario: A user is wearing a smartwatch and in normal working condition. The system continuously collects acceleration and angular velocity data at a sampling frequency of 100Hz. After mean filtering, the sensor data is sent to the reference drift compensation module, which updates the reference point every 200 milliseconds to ensure consistency between the sensor coordinate system and the device's coordinate system. The dynamic gesture segmentation module monitors the acceleration amplitude in real time. When the user performs a wrist-flipping motion, the rotation of the wrist causes the acceleration amplitude to rise to more than 1.2 times the gravitational acceleration within a short period and remain above the dynamic threshold for three consecutive sampling points. The system then determines the start of the gesture and records the start timestamp. The wrist-flipping motion lasts approximately 300 milliseconds. After the motion is completed, the acceleration amplitude gradually decreases and remains below the dynamic threshold for five consecutive sampling points. The system then determines the end of the gesture and records the end timestamp. The feature extraction module processes this 300-millisecond gesture segment, extracting a multi-dimensional feature vector with 3 peaks, a mean peak interval of 120 milliseconds, a main motion direction of 90 degrees clockwise, and a moderate motion amplitude. The gesture recognition module first filters out a subset of candidate templates based on the peak number of 3. Then, it performs Sakoe-Chiba constrained dynamic time warping matching within the subset, calculating the minimum distance score to the wrist-flipping template, and outputs the recognition result as a wrist-flipping action. The control command mapping module calls the screen scrolling interface based on the recognition result to scroll the current interface content clockwise. The system simultaneously updates the reference sample weights of the wrist-flipping template, increasing the weight of the successfully recognized sample to 0.8. The total latency of the entire processing flow, from the start of the gesture to the execution of the control command, is controlled within 50 milliseconds, and the user can hardly perceive any noticeable response delay.

[0064] Example 2

[0065] This embodiment describes a personalized optimization scheme for a specific user group. Based on the basic process of Embodiment 1, this scheme is specifically optimized for the gesture execution characteristics of elderly users or users with movement disorders.

[0066] During the data acquisition phase, considering the slower gesture execution speed and smaller movement amplitude of elderly users, the preset sampling frequency was increased from 100Hz to 200Hz. The higher sampling frequency captures more detailed information during gesture movement, effectively addressing the feature extraction requirements of slow gestures. In the preprocessing phase, the window length of the mean filter was shortened from 5 sampling points to 3 sampling points. A shorter filter window retains higher-frequency motion components, preventing over-smoothing of useful signals during filtering. In the baseline drift compensation phase, the typical value of the adaptive forgetting factor was adjusted from 0.95 to 0.98. A larger forgetting factor makes the estimation results smoother, reducing baseline estimation fluctuations caused by user hand tremors. The baseline update cycle was shortened from 200 milliseconds to 100 milliseconds, allowing for more frequent updates to track baseline drift changes more quickly. In the dynamic gesture segmentation phase, the typical value of the dynamic threshold was reduced from 1.1 times the gravitational acceleration to 1.05 times, to accommodate the smaller movement amplitude of elderly users' gestures. The number of consecutive sampling points exceeding the threshold required to initiate gesture detection was reduced from 3 to 2, lowering the threshold for gesture detection. The number of consecutive sampling points below the threshold required for gesture termination determination has been reduced from 5 to 3 to accommodate the natural pauses that elderly users often make when ending gestures. In the feature extraction stage, the quantization accuracy of the main motion direction has been increased from 1 degree to 0.5 degrees to distinguish gestures with small directional differences. The smoothing parameters of the peak detection algorithm have been adjusted from a Gaussian window length of 5 and a standard deviation of 1.2 to a window length of 3 and a standard deviation of 0.8 to more sensitively capture peak positions. In the recognition stage, the Sakoe-Chiba constrained bandwidth parameter adopts an adaptive strategy to dynamically adjust based on the user's historical performance: if the user's recent recognition success rate is consistently above 95%, the bandwidth is gradually increased to improve recognition accuracy; if the recognition success rate is below 85%, the bandwidth is decreased to expand the search range. The hierarchical recognition strategy uses a soft threshold mechanism for class selection, expanding the candidate range from templates with the same number of peaks to a range of adjacent peaks plus or minus 1, to cover the natural variation in the number of peaks in elderly users' gestures. The preset confidence multiple for confidence assessment has been reduced from 1.5 to 1.2 to reduce the risk of misidentification. The forgetting coefficient α of the personalized adaptation mechanism was adjusted from 0.9 to 0.95, and the weights were updated with more emphasis on the most recently successfully identified samples.

[0067] This optimization scheme enables the system to maintain a high recognition accuracy and response sensitivity even when facing users with slow gesture execution speed and small movement range. The personalized adaptation mechanism allows the system to continuously learn the execution habits of specific users and automatically adjust parameter configurations, achieving a user experience that becomes more accurate the more it is used.

[0068] The above embodiments are merely one of the preferred embodiments of the present invention and should not be used to limit the scope of protection of the present invention. Any modifications or refinements made to the main design concept and spirit of the present invention that are not of substantial significance, but solve the same technical problem as the present invention, should be included within the scope of protection of the present invention.

Claims

1. A gesture interaction control method for a smartwatch, characterized in that, Includes the following steps: S1 collects three-dimensional acceleration and three-dimensional angular velocity signals during the user's gesture movement. Each frame of data includes a timestamp, three-axis acceleration values, and three-axis angular velocity values. S2 performs mean filtering on the collected triaxial acceleration and triaxial angular velocity signals respectively, and converts the data in the local coordinate system of the sensor to the coordinate system of the smartwatch body; S3 estimates and updates the reference point for gesture movement in real time, and uses the adaptive forgetting factor recursive least squares method to estimate the reference drift in order to overcome the cumulative error problem caused by the loosening and shifting of the smartwatch wearing position. S4, based on the compensated acceleration signal amplitude, performs real-time monitoring, and determines the start and end of the gesture by the acceleration amplitude of multiple consecutive sampling points, thereby realizing real-time segmentation of the continuous gesture stream; S5, perform piecewise integration on the angular velocity signal within the gesture segment, calculate the main motion direction and motion amplitude of the gesture, and extract the gesture features of the angular velocity signal to form a multi-dimensional feature vector; S6. Based on the multidimensional feature vector, an optimized dynamic time warping algorithm is used to match and recognize the gesture template library. S7 maps the corresponding gesture to predefined smartwatch control commands based on the recognition results, and executes the corresponding function operations through the smartwatch's processor.

2. The smartwatch gesture interaction control method according to claim 1, characterized in that, In step S1, the three-dimensional acceleration signal and the three-dimensional angular velocity signal are synchronously acquired by the accelerometer and gyroscope built into the smartwatch. A unified hardware timer generates a synchronous trigger signal in each sampling cycle to ensure that the acceleration signal and the angular velocity signal are strictly aligned in the time dimension.

3. The smartwatch gesture interaction control method according to claim 2, characterized in that, In S2, the mean filtering process adopts a sliding window mechanism, and the filter window length is determined based on a trade-off analysis of computational complexity and filtering effect. The coordinate system transformation adopts a static calibration method, which obtains the axial deviation angle between the sensor local coordinate system and the smartwatch body coordinate system through factory calibration, and compensates for it during the data processing stage to ensure the consistency of the two coordinate systems.

4. The smartwatch gesture interaction control method according to claim 3, characterized in that, In S3, the forgetting factor of the adaptive forgetting factor recursive least squares method is set based on the balance between the tracking speed of benchmark changes and the noise suppression capability; the benchmark update cycle eliminates the accumulated error over a long period of use by periodically updating the benchmark reference point, and the compensated benchmark reference point is used to normalize the subsequent feature extraction process.

5. A smartwatch gesture interaction control method according to claim 4, characterized in that, In S4, the specific method for determining the start and end of a gesture is as follows: when the acceleration amplitude of multiple consecutive sampling points exceeds a preset dynamic threshold, the gesture is determined to start; when the acceleration amplitude of multiple consecutive sampling points is lower than the dynamic threshold, the gesture is determined to end. The dynamic threshold is set based on statistical analysis of the acceleration amplitude distribution in static and dynamic states.

6. The smartwatch gesture interaction control method according to claim 5, characterized in that, In S5, the segmented integration of the angular velocity signal adopts the trapezoidal integration method, which divides the duration of the gesture segment into multiple equally spaced time windows, and performs numerical integration on the angular velocity signal within each time window to obtain the angle increment.

7. A smartwatch gesture interaction control method according to claim 6, characterized in that, The multidimensional feature vector extracts the number of peaks, peak intervals, and signal energy distribution of the angular velocity signal simultaneously as statistical features.

8. The smartwatch gesture interaction control method according to claim 7, characterized in that, The dynamic time warping algorithm employs Sakoe-Chiba constraints to limit the search range of the matching path, and adopts a hierarchical recognition strategy. First, it performs broad category filtering based on the number of peaks, and then performs fine matching within each category. The search range is limited to each preset bandwidth sampling point on both sides of the diagonal. The bandwidth parameter is set according to the balance between recognition accuracy and computational efficiency. The larger the bandwidth, the wider the search range but the greater the computational cost; the smaller the bandwidth, the less computation is required. The hierarchical recognition strategy uses the number of peaks for pre-classification, which greatly compresses the number of candidate templates, and then performs fine dynamic time warping matching based on this.

9. A smartwatch gesture interaction control method according to claim 8, characterized in that, The gesture template library contains a variety of predefined gesture templates, and each gesture template contains multiple reference samples to cover the differences in different execution methods. The total storage capacity of the template library is controlled within the preset storage capacity. It supports users to add new gesture templates. Custom templates are generated and added to the template library through the same feature extraction process.

10. A smartwatch gesture interaction control system, characterized in that, For implementing the control method as described in any one of claims 1 to 9, comprising: The data acquisition module is used to simultaneously acquire three-dimensional acceleration signals and three-dimensional angular velocity signals during the user's gesture movement. The preprocessing module is used to perform mean filtering and coordinate transformation on the three-dimensional acceleration signal and the three-dimensional angular velocity signal, respectively. The reference drift compensation module is used to estimate and update the reference point of gesture movement in real time to overcome the cumulative error caused by the loosening and shifting of the smartwatch wearing position; The dynamic gesture segmentation module is used for real-time monitoring based on the compensated acceleration signal amplitude. It determines the start and end of a gesture by comparing the acceleration amplitude of multiple consecutive sampling points with a dynamic threshold, thereby achieving real-time segmentation of a continuous gesture stream. The feature extraction module is used to calculate the main motion direction and motion amplitude of the gesture, and at the same time extract the number of peaks, peak interval and signal energy distribution of the angular velocity signal as statistical features to form a multi-dimensional feature vector; The gesture recognition module is used to perform matching and recognition based on the multi-dimensional feature vector, and combines a hierarchical recognition strategy to first filter the categories based on the number of peaks and then perform fine matching within the categories to output the corresponding gesture category. The control command generation module is used to find the corresponding control command code based on the recognized gesture category and generate the corresponding control command. The execution module is used to execute the functional operations corresponding to the control commands through the processor of the smartwatch.

Images