Swing action detection method based on smart wearable device

Abstract

This invention discloses a method for detecting racket swing motions based on smart wearable devices, addressing the technical problems of current related technologies being unable to effectively capture the rapid dynamic characteristics of ball swing motions, distinguish similar motion patterns, or adapt to differences in user movements. The method includes: collecting motion data when the smart wearable device detects the start of human movement; extracting a complete racket swing motion segment from the motion data based on dual analysis of velocity and electromyography (EMG) signals; extracting multidimensional features from the racket swing motion segment and constructing a multidimensional feature vector; inputting the multidimensional feature vector and the racket swing motion segment into a pre-trained ball motion recognition model for ball swing motion recognition, and outputting ball recognition results and swing type prediction results.

Description

Technical Field

[0001] This invention relates to the field of smart wearable device technology, and in particular to a method for detecting racket swing motion based on a smart wearable device, a device for detecting racket swing motion based on a smart wearable device, a smart wearable device, and a storage medium. Background Technology

[0002] Currently available smart wearable devices (such as smartwatches, smart bracelets, and smart rings) have the following main shortcomings in sports mode recognition, especially for sports requiring precise racket swings, such as badminton, table tennis, and tennis:

[0003] 1. Missing Functionality or Insufficient Accuracy: Most smart wearable devices can only recognize common sports such as running and swimming. They offer very little support for specialized monitoring functions for racket-based sports such as badminton, table tennis, and tennis. Alternatively, they can only perform rough counts and cannot identify specific swing types or assess the quality of the movements.

[0004] 2. Limitations of Sensor Configuration and Algorithms: Traditional devices often use a single sensor. Their sampling rate, accuracy, and crucial algorithm optimization are insufficient to capture fast, complex, and multi-dimensional data. Furthermore, single-sensor data has limited dimensions, weak anti-interference capabilities, and difficulty in distinguishing similar actions.

[0005] 3. Reliance on external devices: Some related technologies require additional equipment such as sensors installed on the racket or electromyography (EMG) sensors worn on specific parts of the arm. This significantly increases the barrier to entry, cost, and inconvenience for users of smart wearable devices, hindering their widespread adoption.

[0006] 4. Poor versatility and insufficient algorithm robustness: A few products support swing detection for a specific type of ball. However, this is limited to optimization for a single ball. Their algorithm models have weak generalization ability, making it difficult to accurately identify and distinguish multiple swing types using the same device. Furthermore, they are susceptible to individual differences and environmental interference.

[0007] The aforementioned shortcomings are mainly due to the limited sensor information, rudimentary algorithm models, and insufficient feature data extraction capabilities. These issues result in core pain points for current smart wearable devices in racket-based sports monitoring, including low recognition accuracy, limited functionality, poor versatility, and unsatisfactory user experience. Particularly for sports like badminton, table tennis, and tennis, traditional technologies struggle to effectively capture the rapid dynamic characteristics of racket swings, differentiate similar movement patterns, and adapt to the varying movements of different users, thus limiting their application effectiveness in real-world scenarios. Summary of the Invention

[0008] This invention provides a method for detecting racket swing motions based on a smart wearable device, a device for detecting racket swing motions based on a smart wearable device, a smart wearable device, and a storage medium, to solve or partially solve the technical problems of current related technologies being unable to effectively capture the rapid dynamic characteristics of racket swing motions in ball games, unable to distinguish similar motion patterns, and unable to adapt to the differences in motions of different users.

[0009] This invention provides a method for detecting racket swing motions based on a smart wearable device, the method comprising:

[0010] When a smart wearable device detects that a person has started to move, it collects motion data;

[0011] The motion data is subjected to racket swing motion extraction based on dual analysis of velocity and electromyography signals to obtain a complete racket swing motion segment;

[0012] Multidimensional features are extracted from the swing motion segment, and a multidimensional feature vector is constructed.

[0013] The multidimensional feature vector and the swing motion fragment are input into a pre-trained ball motion recognition model to recognize ball swing motions, and the ball recognition result and swing type prediction result are output.

[0014] Optionally, the motion data includes acceleration and angular velocity data and electromyography (EMG) signals; the extraction of a complete swing motion segment from the motion data based on dual analysis of velocity and EMG signals includes:

[0015] The acceleration and angular velocity data and the electromyographic signal are time-stamped and then filtered to obtain calibrated acceleration and angular velocity data and calibrated electromyographic signals.

[0016] The time-domain integral and frequency-domain energy calculations are performed on the calibrated electromyographic signal to obtain the energy characteristics of the electromyographic signal;

[0017] Based on the calibrated acceleration and angular velocity data and the electromyographic signal energy characteristics, a comprehensive swing judgment is made to determine the starting point and ending point of a complete swing motion.

[0018] The acceleration time sequence and electromyography time sequence from the start point to the end point of the swing are extracted from the calibrated acceleration and angular velocity data and the calibrated electromyography signal to form a complete swing motion segment.

[0019] Optionally, the smart wearable device has a built-in A+G sensor module and an electromyography (EMG) sensor; when the smart wearable device detects that the human body has started to move, it collects motion data, including:

[0020] When an acceleration signal is detected by the A+G sensor module, it is determined that the human body has started to move. Using a time-series recording method, the A+G sensor module collects and records acceleration and angular velocity data, and the electromyography sensor collects and records the electromyographic signals generated during human muscle activity.

[0021] Optionally, the step of comprehensively judging the swing based on the calibrated acceleration and angular velocity data and the electromyographic signal energy characteristics to determine the starting and ending points of a complete swing motion includes:

[0022] Based on the calibrated acceleration and angular velocity data, the composite acceleration magnitude and composite angular velocity magnitude at each moment in the recorded time series are calculated one by one in chronological order.

[0023] During the calculation process, if the combined acceleration modulus and the combined angular velocity modulus exceed the preset dynamic threshold and meet the correlation coefficient condition, and the electromyographic signal energy value is determined to be within the preset range based on the electromyographic signal energy characteristics, then the first time point is marked as the starting point of a complete swing motion.

[0024] Continue the calculation. If there is a second time point when the combined acceleration modulus and the combined angular velocity modulus are lower than the preset dynamic threshold and can be maintained for a preset duration, and the electromyographic signal energy value is within the preset normal range based on the electromyographic signal energy characteristics, then mark the second time point as the swing end point corresponding to the swing start point.

[0025] Optionally, the extracted multidimensional features include acceleration multidimensional features, angular velocity multidimensional features, and electromyographic signal energy features; the electromyographic signal energy features include the time-domain integral value of the electromyographic signal and the energy proportion of the electromyographic signal in different frequency domains.

[0026] Optionally, the pre-trained ball action recognition model includes a neural network model pre-trained based on ball recognition and several LSTM sub-models pre-trained based on swing action recognition; the step of inputting the multi-dimensional feature vector and the swing action fragment into the pre-trained ball action recognition model for ball swing action recognition, and outputting ball recognition results and swing type prediction results, includes:

[0027] The multidimensional feature vector is input into the neural network model to perform ball recognition and obtain the ball recognition result.

[0028] Determine the target LSTM sub-model corresponding to the ball identification result, call the target LSTM sub-model to perform swing type identification on the swing action segment, and output the swing type prediction result.

[0029] Optionally, the smart wearable device is communicatively connected to a smart terminal; the method further includes:

[0030] Based on the calibrated acceleration and angular velocity data obtained after calibration and filtering, the peak swing velocity and peak swing acceleration are calculated respectively.

[0031] Based on the peak swing acceleration, the hitting power index is estimated, and the hitting power index is corrected based on the electromyographic signal intensity to obtain the corrected hitting power index.

[0032] The ball identification result, the swing type prediction result, the peak swing speed, the peak swing acceleration, the corrected hitting power index, and the current swing count are packaged and sent to the smart terminal.

[0033] The present invention also provides a swing motion detection device based on a smart wearable device, the device comprising:

[0034] The data acquisition unit is used to collect motion data when the smart wearable device detects that the human body has started to move;

[0035] The racket swing motion extraction unit is used to extract the racket swing motion based on the motion data through dual analysis of velocity and electromyography signals, so as to obtain a complete racket swing motion segment.

[0036] A multidimensional feature extraction unit is used to extract multidimensional features from the swing motion segment and construct a multidimensional feature vector;

[0037] The ball swing action recognition unit is used to input the multi-dimensional feature vector and the swing action segment into a pre-trained ball action recognition model to recognize the ball swing action, and output the ball recognition result and the swing type prediction result.

[0038] The present invention also provides a smart wearable device, the smart wearable device including a processor and a memory:

[0039] The memory is used to store program code and transmit the program code to the processor;

[0040] The processor is used to execute the swing motion detection method based on a smart wearable device as described above, according to the instructions in the program code.

[0041] The present invention also provides a computer-readable storage medium for storing program code for executing the racket swing action detection method based on a smart wearable device as described in any of the preceding claims.

[0042] As can be seen from the above technical solutions, the present invention has the following advantages:

[0043] This paper presents a method for detecting racket swing motions based on smart wearable devices. When the smart wearable device detects the start of human movement, it collects motion data as the foundation for subsequent analysis. The motion data is then processed to extract the racket swing motion based on both velocity and electromyography (EMG) signals, yielding a complete swing motion segment. A dedicated swing motion extraction and fusion algorithm is designed for the collected velocity-related and EMG signal-related data, effectively integrating complementary information from different sensor sources. This significantly improves the accuracy and anti-interference capability of motion state estimation in complex swing motions. Next, multi-dimensional features are extracted from the swing motion segment, and a multi-dimensional feature vector is constructed. Through carefully designed multi-dimensional feature engineering, highly discriminative multi-dimensional motion features are extracted from the fused swing motion data, and a multi-dimensional feature vector is constructed. This allows for more accurate and effective prediction results in subsequent ball and swing type recognition steps. The multi-dimensional feature vector and the swing motion segment are input into a pre-trained ball motion recognition model for ball and swing motion recognition, outputting ball recognition results and swing type prediction results. Therefore, by introducing a specially pre-trained recognition model based on ball swing motion recognition, and leveraging its powerful nonlinear fitting and pattern recognition capabilities, this approach achieves accurate recognition of various ball sports and fine classification of different swing motion types within the same ball sport. This technical solution effectively addresses the challenges of current related technologies in effectively capturing the rapid dynamic characteristics of ball swing motions and distinguishing similar motion patterns. Furthermore, based on the recognition model's accurate recognition of different ball sports and its fine classification of different swing motion types within the same ball sport, the model can better identify subtle differences in motion even when recognizing swing motions from different users, adapting to individual user motion variations and providing accurate ball sports and swing motion type predictions. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a block diagram illustrating the principle of racket swing detection based on a smart wearable device.

[0046] Figure 2 A flowchart illustrating the steps of a racket swing motion detection method based on a smart wearable device;

[0047] Figure 3This is a schematic diagram of the overall process of a racket swing motion detection method based on smart wearable devices;

[0048] Figure 4 This is a structural block diagram of a racket swing motion detection device based on a smart wearable device. Detailed Implementation

[0049] This invention provides a method for detecting racket swing motions based on a smart wearable device, a device for detecting racket swing motions based on a smart wearable device, a smart wearable device, and a storage medium, to solve or partially solve the technical problems of current related technologies being unable to effectively capture the rapid dynamic characteristics of racket swing motions in ball games, unable to distinguish similar motion patterns, and unable to adapt to the differences in motions of different users.

[0050] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0051] As an example, current smart wearable devices on the market suffer from several shortcomings in sports pattern recognition, especially for sports like badminton, table tennis, and tennis that require precise swinging motions. These shortcomings include missing functions or insufficient accuracy, limitations in sensor configuration and algorithms, reliance on external devices, poor versatility, and insufficient algorithm robustness.

[0052] The aforementioned shortcomings are mainly due to the limited range of sensor information, rudimentary algorithm models, and insufficient feature data extraction capabilities. These issues result in core pain points for current smart wearable devices in monitoring racket-based sports, including low recognition accuracy, limited functionality, poor versatility, and unsatisfactory user experience. Particularly for sports like badminton, table tennis, and tennis, traditional technologies struggle to effectively capture the rapid dynamic characteristics of racket swings, differentiate similar movement patterns, and adapt to the varying movements of different users, thus limiting their application effectiveness in real-world scenarios.

[0053] Therefore, one of the core inventive points of this invention is to provide a smart wearable device that integrates an accelerometer and gyroscope sensor (A+G) and an electromyography (EMG) sensor, and achieves fine motion detection and analysis of different ball game swings through an optimized fusion algorithm, as well as a ball game swing motion detection method based on the smart wearable device. On one hand, by proposing the integration of two physically diagonally arranged A+G sensors (or multiple A+G sensors as needed) within the smart wearable device, and simultaneously integrating an EMG sensor, differentiated and complementary motion information is obtained, breaking through the limitations of single-sensor information dimensions and providing a hardware foundation for subsequent high-precision algorithm analysis. On the other hand, a dedicated spatiotemporal alignment and fusion algorithm is designed for the A+G sensor and EMG sensor data, which can effectively integrate the complementary information of multiple sensors, significantly improving the accuracy and anti-interference capability of motion state estimation in complex swing motions. Furthermore, through meticulously designed multidimensional feature engineering, highly discriminative motion features are extracted from sensor fusion data. A deep learning model pre-trained specifically for ball swing motion recognition is introduced, leveraging its powerful nonlinear fitting and pattern recognition capabilities to achieve accurate recognition of various ball sports, fine classification of different swing motion types within the same ball sport, and accurate calculation of key motion parameters.

[0054] Reference Figure 1 The diagram illustrates a principle block diagram of a racket swing action detection based on a smart wearable device, according to an embodiment of the present invention.

[0055] It is understood that the smart wearable device proposed in the embodiments of the present invention can be any smart device that supports internally integrated sensors and communication methods such as wireless / Bluetooth / NFC, including smartwatches / bracelets / rings. Specifically, Figure 1 This paper mainly illustrates the hardware execution layer architecture for racket swing detection based on smart wearable devices.

[0056] The A+G sensor is an integrated sensor that combines an accelerometer (ACC) and a gyroscope (GRY). In some embodiments, two A+G integrated sensors can be placed diagonally to form a dual A+G sensor module. In other embodiments, multiple A+G integrated sensors (three or more) can be used, and their placement can be adjusted according to actual conditions to form a multi-A+G sensor module. It is understood that this invention does not impose limitations on these aspects. For ease of understanding, this invention uses a dual A+G sensor module as an example for illustration.

[0057] The electromyography (EMG) sensor has three electrodes: ECGN (Electrocardiogram Noise), ECGP (Electrocardiogram Processing), and RLD (Rolling Data). The EMG sensor is located on the sub-board of the smart wearable device, and its three electrodes are integrated into the rear lens of the device. The rear lens has a total of eight electrodes; the EMG electrode positions are not fixed and can be placed as needed.

[0058] The processor simultaneously configures a dual A+G sensor module and an electromyography (EMG) sensor via a bus. The dual A+G sensor module is used to acquire acceleration and angular velocity data. The EMG sensor is used to acquire electromyographic signals generated by human muscle activity. In practical applications, a hardware interrupt signal triggers synchronous sampling of the dual A+G sensor module and the EMG sensor. The raw data from both channels is timestamped and calibrated. A 5th-order Butterworth low-pass filter is used to remove high-frequency noise, and a high-pass filter is used to eliminate gravitational interference.

[0059] Reference Figure 2 The diagram illustrates a flowchart of a swing motion detection method based on a smart wearable device according to an embodiment of the present invention, which may specifically include the following steps:

[0060] Step 201: When the smart wearable device detects that the human body has started to move, it collects motion data;

[0061] In specific implementation, when the smart wearable device detects that a human body has begun to move, it initiates the collection of motion data. As discussed above, the smart wearable device provided in this embodiment of the invention has a built-in A+G sensor module (which can be a dual A+G sensor module or a multi-A+G sensor module; this invention uses a dual A+G sensor module as an example) and an electromyography (EMG) sensor. Specifically, the process of collecting motion data when the smart wearable device detects that a human body has begun to move can include: when an acceleration signal is detected by the dual A+G sensor module, it is determined that the human body has begun to move; using a time-series recording method, acceleration and angular velocity data are collected and recorded by the dual A+G sensor module, and electromyography signals generated during muscle activity are collected and recorded by the EMG sensor; the collected acceleration and angular velocity data and EMG signals are integrated as motion data. In other words, the motion data collected in this step is essentially time-series data related to acceleration / angular velocity and EMG signals.

[0062] Step 202: Extract the swing motion from the motion data based on dual analysis of velocity and electromyography signals to obtain a complete swing motion segment;

[0063] Step 202 pertains to the signal processing algorithm layer. In step 202, the motion data acquired in the preceding steps is analyzed using both velocity and electromyography (EMG) signals to extract the racket swing motion, resulting in a complete swing motion segment. This mainly includes key steps such as timestamp calibration, filtering, marking the start and end points of the swing, and extracting the swing motion segment.

[0064] The first step is timestamp calibration. Timestamp calibration is performed on the dual-channel raw data (accelerometer and angular velocity data) from the A+G sensor and the raw data (electromyography signal) from the electromyography sensor to ensure that the time deviation of the three data is ≤1ms.

[0065] Secondly, filtering is performed. After calibration, on the one hand, a 5th-order Butterworth low-pass filter is used to remove high-frequency noise from the acceleration and angular velocity data of the A+G sensor, and a high-pass filter is used to eliminate gravitational interference. On the other hand, the electromyographic (EMG) signal is filtered to remove noise interference and retain the effective signal. Then, time-domain integration and frequency-domain energy calculation are performed on the filtered EMG signal to obtain the EMG signal energy characteristics. The EMG signal energy characteristics are multi-dimensional feature data of the EMG signal, including the time-domain integral value of the EMG signal and the energy proportion of the EMG signal in different frequency domains (EMG signal energy value).

[0066] For the swing start and end point marking process, based on the chronological order of the acquisition of relevant time series motion data, the synthetic acceleration modulus and synthetic angular velocity modulus of the A+G sensors are calculated one by one at each time point in the recorded time series. During the calculation process, the electromyographic signal energy value obtained by time-domain integration and frequency-domain energy calculation is combined. When the value calculated by the A+G sensor data (i.e., the synthetic acceleration modulus and synthetic angular velocity modulus of the A+G sensors mentioned above) exceeds the preset dynamic threshold and the correlation coefficient of the dual A+G sensor data is >0.8, and the electromyographic signal energy value also shows a significant change (such as when the electromyographic signal energy value is within the preset change range), the corresponding time point is marked as the swing start point. When the value calculated by the dual A+G sensor data falls back below the preset dynamic threshold and remains below it for 100ms, and the electromyographic signal energy value also returns to a normal level (such as when the electromyographic signal energy value is within the preset normal range), the corresponding time point is marked as the swing end point.

[0067] Based on the start and end points of the swing, corresponding time series data are extracted from the calibrated acceleration and angular velocity data and the calibrated electromyography (EMG) signals (acceleration time series is extracted from the calibrated acceleration and angular velocity data, and EMG signal time series is extracted from the calibrated EMG signals), and recorded as a swing action segment of a complete swing action.

[0068] Based on the preceding discussion, motion data mainly includes acceleration and angular velocity data, as well as electromyography (EMG) signals. In the specific implementation, the process of extracting a complete swing motion segment from the motion data based on dual analysis of velocity and EMG signals can include: timestamping the acceleration and angular velocity data and EMG signals, and then filtering them to obtain calibrated acceleration and angular velocity data and calibrated EMG signals; performing time-domain integration and frequency-domain energy calculation on the calibrated EMG signals to obtain EMG signal energy characteristics; making a comprehensive swing judgment based on the calibrated acceleration and angular velocity data and EMG signal energy characteristics to determine the start and end points of a complete swing motion; and extracting the acceleration time series and EMG signal time series from the start and end points of the swing motion from the calibrated acceleration and angular velocity data and the calibrated EMG signals as a complete swing motion segment.

[0069] Furthermore, the specific implementation process for determining the start and end points of a complete swing motion by comprehensively judging the swing based on calibrated acceleration and angular velocity data and electromyographic signal energy characteristics can include: calculating the synthetic acceleration magnitude and synthetic angular velocity magnitude at each time point in the recorded time series in chronological order based on calibrated acceleration and angular velocity data; during the calculation, if there is a first time point where the synthetic acceleration magnitude and synthetic angular velocity magnitude exceed a preset dynamic threshold and meet the correlation coefficient condition, and the electromyographic signal energy value is within a preset range of change based on the electromyographic signal energy characteristics, then the first time point is marked as the start point of a complete swing motion; the calculation continues, and if there is a second time point where the synthetic acceleration magnitude and synthetic angular velocity magnitude are lower than the preset dynamic threshold and can be maintained for a preset duration, and the electromyographic signal energy value is within a preset normal range based on the electromyographic signal energy characteristics, then the second time point is marked as the end point of the swing corresponding to the start point.

[0070] Step 203: Extract multi-dimensional features from the swing motion segment and construct a multi-dimensional feature vector;

[0071] After obtaining a fragment of the swing motion, multidimensional features can be extracted from the fragment, and a multidimensional feature vector can be constructed. In some embodiments, the multidimensional features extracted in this step can mainly include multidimensional acceleration features, multidimensional angular velocity features, and electromyographic (EMG) signal energy features. The EMG signal energy features, which are one of the important improvements of this invention, can include the time-domain integral value of the EMG signal and the energy proportion of the EMG signal in different frequency domains.

[0072] Specifically, step 203 pertains to the feature extraction layer. This step primarily extracts and constructs a 32-dimensional feature vector. When extracting the 32-dimensional feature vector from the racket swing motion segment, in addition to the peak / valley values ​​of triaxial acceleration from the dual A+G sensors, peak / valley values ​​of angular velocity, motion duration, and impact factor, as well as the energy proportions of the 0-10Hz, 10-20Hz, and 20-50Hz frequency bands extracted by fast Fourier transform of the acceleration signal, the integral of the absolute value of the triaxial acceleration difference, the standard deviation of the angular velocity ratio, and the rotational angular acceleration calculated based on the position difference, features related to electromyography (EMG) signals are added. These include the time-domain integral value of the EMG signal and the energy proportions of the EMG signal in different frequency domains (0-10Hz, 10-20Hz, 20-50Hz, etc.).

[0073] Therefore, this step proposes a multi-dimensional feature extraction method suitable for swing type classification. By extracting a high-discrimination feature set containing 32-dimensional time-domain, frequency-domain, and dual-sensor coupling features from dual A+G sensor and electromyography (EMG) sensor data, especially original features such as the integral of the acceleration difference between the two sensors and the standard deviation of the angular velocity ratio, as well as EMG signal-related features, the feature discrimination is significantly improved compared to traditional single-sensor feature sets, providing key support for the accurate and effective classification of the model.

[0074] Step 204: Input the multidimensional feature vector and the swing action segment into the pre-trained ball action recognition model to perform ball swing action recognition, and output the ball recognition result and the swing type prediction result.

[0075] Step 204 pertains to the model inference layer's processing flow. It mainly includes: inputting multi-dimensional feature vectors and swing motion fragments into a pre-trained ball motion recognition model to perform ball swing motion recognition, and outputting ball recognition results and swing type prediction results. Ball swing motion recognition primarily includes ball recognition and swing type recognition.

[0076] Specifically, the general implementation process for ball recognition is as follows: A feature vector containing 32 dimensions (incorporating electromyographic signal features) is input into a pre-trained 1D-CNN model (1-Dimensional Convolutional Neural Network, containing 3 convolutional layers and 2 fully connected layers). The model can output probability values ​​for three ball types: badminton, table tennis, and tennis. The ball label corresponding to the highest probability (confidence threshold ≥ 0.85) is taken as the ball recognition result. The confidence threshold is used to filter out some prediction boxes with low confidence. If the confidence threshold is set too high, some true foreground targets may be filtered out. If the confidence threshold is set too low, too many background prediction boxes may be retained, thus affecting subsequent prediction results. In this embodiment, the confidence threshold is set to a suitable 0.85, which neither filters out true target classification labels nor ball classification labels with low confidence.

[0077] The general implementation process of swing type recognition is as follows: Based on the ball label, the corresponding LSTM sub-model for the target ball is called from multiple LSTM (Long Short-Term Memory) sub-models pre-trained based on different swing actions of the same ball. The acceleration time series (length 200ms) and electromyography signal time series (length 200ms) of the swing action segment are input. The bidirectional LSTM network outputs the swing type probability, and the maximum value is taken as the classification result.

[0078] For example, suppose we are performing ball type recognition and corresponding swing type recognition for three ball sports: badminton, table tennis, and tennis. Ball type recognition yields the corresponding ball type result, such as identifying it as badminton. After initial model training, there are three LSTM sub-models pre-trained based on different swing actions in badminton, table tennis, and tennis (specifically, a badminton LSTM sub-model pre-trained based on 5 categories: forehand / backhand / smash / drop / lift; a table tennis LSTM sub-model pre-trained based on 3 categories: forehand attack / backhand push / smash; and a tennis LSTM sub-model pre-trained based on 3 categories: forehand hit / backhand slice / volley; these three sub-models are obtained by adaptively adjusting parameters for different motion data, following the conventional publicly available LSTM model framework). At this point, we directly call the badminton LSTM sub-model to perform swing type recognition, obtaining the probability prediction results for each of the 5 categories: forehand / backhand / smash / drop / lift, and taking the swing type classification result with the highest probability as the final swing type prediction result.

[0079] Based on the foregoing discussion, the pre-trained ball action recognition model referred to in this embodiment of the invention may further include a neural network model pre-trained based on ball recognition and several LSTM sub-models pre-trained based on swing action recognition. In a specific implementation, the process of inputting multi-dimensional feature vectors and swing action segments into the pre-trained ball action recognition model to perform ball swing action recognition and output ball recognition results and swing type prediction results may include: inputting multi-dimensional feature vectors into the neural network model to perform ball recognition and obtain ball recognition results; determining the target LSTM sub-model corresponding to the ball recognition results; calling the target LSTM sub-model to perform swing type recognition on the swing action segments and outputting swing type prediction results.

[0080] Therefore, based on multi-source fusion features (including motion parameters such as acceleration and angular velocity, as well as electromyographic signal parameters) and optimized training deep learning methods, an innovative architecture combining a neural network model and an LSTM sub-model was constructed to achieve accurate recognition of various ball sports such as badminton, table tennis, and tennis, as well as fine classification of specific ball swing types. Compared with traditional technologies, the recognition model provided in this embodiment of the invention has achieved significant improvements in accuracy for both multi-ball sports recognition and single-ball swing type recognition. Furthermore, the combination of real-time data acquisition and processing based on multi-source sensors, along with the model's rapid and effective recognition, gives the model stronger real-time performance and robustness.

[0081] Furthermore, this invention further evaluates the quality of the swing motion and enables interaction between the smart wearable device and an external smart terminal at the data interaction level. Its basic implementation principle is as follows: The peak swing speed is calculated based on the integral of fused angular velocity data; the hitting power index is estimated by combining the peak swing acceleration (i.e., force-based estimation based on Newton's second law, using the magnitude of the force or a standardized value of the force as the hitting power index); and the hitting power index is corrected by considering the intensity of electromyographic signals (i.e., correlation analysis between electromyographic signal intensity and the hitting power index). The ball type identification result and swing type prediction result obtained through the preceding identification steps, along with the current swing count, peak swing speed, peak swing acceleration, and the corrected hitting power index, are packaged and pushed to the screen of a smart terminal (such as a mobile phone, smart tablet, etc.) for real-time display via Bluetooth or wireless push. Additionally, it can be set to synchronize to the accompanying APP of the smart wearable device every 30 seconds via Bluetooth or wireless push.

[0082] Among them, the intensity of electromyography (EMG) signals is considered to correct the hitting power index. This can be done by: based on the energy ratio of EMG signals in different frequency domains, when EMG_norm < 10 (non-power state), the forced power coefficient = 0.5 to avoid over-correction; at the same time, when the swing speed < 2m / s (ineffective swing), the basic hitting power index is directly output.

[0083] Based on the foregoing discussion, in some embodiments, the smart wearable device can also communicate with smart terminals (such as mobile phones, smart tablets, etc.) via Bluetooth / Wi-Fi. After obtaining the ball identification result and swing type prediction result through the ball swing action recognition steps provided in the preceding embodiments, the peak swing speed and peak swing acceleration can be calculated based on the calibrated acceleration and angular velocity data obtained after calibration and filtering. Then, based on the peak swing acceleration, the hitting power index is estimated, and a correlation correction based on electromyographic signal intensity is applied to the hitting power index to obtain the corrected hitting power index. Finally, the ball identification result, swing type prediction result, peak swing speed, peak swing acceleration, corrected hitting power index, and current swing count are packaged and sent to the smart terminal.

[0084] This invention provides a smart wearable device that integrates dual accelerometer and gyroscope sensors (A+G) and electromyography (EMG) sensors, and achieves fine motion detection and analysis of different ball games' racket swings through optimized fusion algorithms, as well as a method for detecting racket swing motions based on the smart wearable device. On one hand, by integrating two physically diagonally positioned A+G sensors (or multiple A+G sensors as needed) within the smart wearable device, along with an EMG sensor, differentiated and complementary motion information is obtained, overcoming the limitations of single-sensor information dimensions and providing a hardware foundation for subsequent high-precision algorithm analysis. Thus, detection is achieved solely through the A+G and EMG sensors integrated into the smart wearable device itself, eliminating the need for users to wear any additional sensors on the racket or other parts of the body. Compared to solutions requiring additional sensors, the user barrier is significantly lowered. On the other hand, by combining neural network models and LSTM models for racket swing motion recognition, subtle features of the swing motion can be captured, achieving fine recognition of specific ball games and swing types. The recognition model constructed in this invention can effectively identify and distinguish various racket-swing sports such as badminton, table tennis, and tennis. Compared to single-ball monitoring products, the range of applicable sports is significantly expanded, and a higher ball recognition accuracy is achieved. Even for a single ball, the recognition model can identify multiple swing types with a high average classification accuracy. Compared to products that can only roughly count, this significantly increases the dimensionality of sports data, providing a data foundation for motion analysis. Simultaneously, dual-sensor data fusion provides redundant information and more dimensional features. Combined with subsequent optimized algorithm model recognition, it effectively reduces interference from factors such as single-sensor noise and minor changes in wearing position, significantly lowering the false positive rate of swing event detection and improving detection stability in complex sports scenarios. Furthermore, through further analysis and calculation, this invention can output key parameters such as swing speed, force estimation, and motion type proportion, helping users scientifically evaluate exercise effects. The algorithm-driven motion quality analysis can provide personalized training suggestions, improving the efficiency of users' skill improvement.

[0085] For better explanation, refer to Figure 3 This diagram illustrates the overall flow of a racket-swinging motion detection method based on a smart wearable device, as provided in an embodiment of the present invention. It should be noted that this embodiment only provides a brief overview of the general flow of racket-swinging motion detection based on a smart wearable device. The specific implementation process of each step can be understood by referring to the relevant content in the foregoing embodiments, and will not be elaborated upon here. It is understood that the present invention does not impose any limitations on this.

[0086] Step 301: When an acceleration signal is detected, it is determined that the human body has started to move. The acceleration and angular velocity data are collected and recorded using a time-series recording method. At the same time, the electromyographic signals generated by the human muscles during muscle activity are collected and recorded.

[0087] Step 302: Time-stamp the acceleration and angular velocity data and electromyography signals, and filter them respectively to obtain calibrated acceleration and angular velocity data and calibrated electromyography signals;

[0088] Step 303: Perform time-domain integration and frequency-domain energy calculation on the calibrated electromyography (EMG) signal to obtain the EMG signal energy characteristics. Based on the calibrated acceleration and angular velocity data and the EMG signal energy characteristics, make a comprehensive swing judgment to determine the swing start point and swing end point of a complete swing action.

[0089] Step 304: Extract the acceleration time series and electromyography signal time series from the start point to the end point of the swing from the motion data as a complete swing action segment, and perform multi-dimensional feature extraction on the swing action segment to construct a multi-dimensional feature vector;

[0090] Step 305: Input the multidimensional feature vector into the pre-trained neural network model to perform ball recognition, obtain the ball recognition result, determine the target LSTM sub-model corresponding to the ball recognition result from multiple pre-trained LSTM sub-models, call the target LSTM sub-model to perform swing type recognition on the swing action segment, and output the swing type prediction result.

[0091] Step 306: Based on the calibrated acceleration and angular velocity data, calculate the peak swing speed and peak swing acceleration respectively. Based on the peak swing acceleration, estimate the hitting power index and perform correlation correction based on electromyographic signal intensity to obtain the corrected hitting power index.

[0092] Step 307: Pack the ball type identification result, swing type prediction result, peak swing speed, peak swing acceleration, corrected hitting power index, and current swing count into a package and send it to the smart terminal.

[0093] Reference Figure 4 The diagram illustrates a structural block diagram of a swing motion detection device based on a smart wearable device according to an embodiment of the present invention, which may specifically include:

[0094] The data acquisition unit 401 is used to collect motion data when the smart wearable device detects that the human body has started to move;

[0095] The swing motion extraction unit 402 is used to extract the swing motion based on the motion data using both velocity and electromyographic signals, and obtain a complete swing motion segment.

[0096] The multidimensional feature extraction unit 403 is used to extract multidimensional features from the swing motion segment and construct a multidimensional feature vector;

[0097] The ball swing action recognition unit 404 is used to input the multi-dimensional feature vector and the swing action segment into a pre-trained ball action recognition model to perform ball swing action recognition, and output ball recognition results and swing type prediction results.

[0098] In one optional embodiment, the motion data includes acceleration and angular velocity data and electromyographic signals; the racket swing motion extraction unit 402 includes:

[0099] The signal processing unit is used to perform time-stamp calibration on the acceleration and angular velocity data and the electromyographic signal, and to perform filtering processing on them respectively to obtain calibrated acceleration and angular velocity data and calibrated electromyographic signal.

[0100] An electromyography (EMG) signal calculation unit is used to perform time-domain integration and frequency-domain energy calculation on the calibrated EMG signal to obtain the energy characteristics of the EMG signal.

[0101] The integrated swing judgment unit is used to make an integrated swing judgment based on the calibrated acceleration and angular velocity data and the electromyographic signal energy characteristics, and to determine the swing start point and swing end point of a complete swing action;

[0102] The time series extraction unit is used to extract the acceleration time series and electromyography signal time series from the calibrated acceleration and angular velocity data and the calibrated electromyography signal from the swing start point to the swing end point, as a complete swing motion segment.

[0103] In one optional embodiment, the smart wearable device integrates an A+G sensor module and an electromyography (EMG) sensor; the data acquisition unit 401 is specifically used for:

[0104] When an acceleration signal is detected by the A+G sensor module, it is determined that the human body has started to move. Using a time-series recording method, the A+G sensor module collects and records acceleration and angular velocity data, and the electromyography sensor collects and records the electromyographic signals generated during human muscle activity.

[0105] In one optional embodiment, the integrated swing judgment unit includes:

[0106] The synthetic modulus calculation unit is used to calculate the synthetic acceleration modulus and synthetic angular velocity modulus at each time point in the recorded time series in chronological order, based on the calibrated acceleration and angular velocity data.

[0107] The swing start point marking unit is used to mark the first moment point as the swing start point of a complete swing action if, during the calculation process, the combined acceleration modulus and the combined angular velocity modulus exceed the preset dynamic threshold and meet the correlation coefficient condition, and the electromyographic signal energy value is within the preset range based on the electromyographic signal energy characteristics.

[0108] The swing end point marking unit is used to continue the calculation. If there is a second time point when the combined acceleration modulus and the combined angular velocity modulus are lower than the preset dynamic threshold and can be maintained for a preset duration, and the electromyographic signal energy value is judged to be within the preset normal range based on the electromyographic signal energy characteristics, then the second time point is marked as the swing end point corresponding to the swing start point.

[0109] In one optional embodiment, the extracted multidimensional features include acceleration multidimensional features, angular velocity multidimensional features, and electromyographic signal energy features; the electromyographic signal energy features include the time-domain integral value of the electromyographic signal and the energy proportion of the electromyographic signal in different frequency domains.

[0110] In one optional embodiment, the pre-trained ball action recognition model includes a neural network model pre-trained based on ball recognition and several LSTM sub-models pre-trained based on racket swing action recognition; the ball swing action recognition unit 404 includes:

[0111] A ball recognition unit is used to input the multidimensional feature vector into the neural network model to perform ball recognition and obtain ball recognition results.

[0112] The swing type recognition unit is used to determine the target LSTM sub-model corresponding to the ball type recognition result, call the target LSTM sub-model to perform swing type recognition on the swing action segment, and output the swing type prediction result.

[0113] In one optional embodiment, the smart wearable device is communicatively connected to a smart terminal; the device further includes:

[0114] The peak velocity calculation unit is used to calculate the peak swing velocity and peak swing acceleration based on the calibrated acceleration and angular velocity data obtained after calibration and filtering.

[0115] The corrected hitting power index calculation unit is used to estimate the hitting power index based on the peak value of the swing acceleration, and to perform correlation correction on the hitting power index based on the electromyographic signal intensity to obtain the corrected hitting power index.

[0116] The data transmission unit is used to package and send the ball identification result, the swing type prediction result, the peak swing speed, the peak swing acceleration, the corrected hitting power index, and the current swing count to the smart terminal.

[0117] As the device embodiment is basically similar to the method embodiment, it is described in a relatively simple way. For relevant details, please refer to the description of the method embodiment above.

[0118] It should be noted that intelligent systems that adopt or include similar hardware architectures and related algorithms, including but not limited to the expansion from single-task to multi-task basic scenarios, such as sensor data + image, sensor data + audio, etc., as well as the vertical industry expansion from a single industry to the entire industry, such as security, medical, education, and industrial fields, all fall within the protection scope of the technical solutions proposed in the embodiments of this invention.

[0119] It should be noted that, in order to enable those skilled in the art to better distinguish data of the same type but with different actual meanings, the embodiments of the present invention use "first" and "second" to distinguish and describe some technical features. "First" and "second" are only used to distinguish data and have no other special meaning. It is understood that the present invention does not impose any limitations on them.

[0120] This invention also provides a smart wearable device, which includes a processor and a memory:

[0121] The memory is used to store program code and transfer the program code to the processor;

[0122] The processor is used to execute the swing motion detection method based on a smart wearable device according to the instructions in the program code of any embodiment of the present invention.

[0123] This invention also provides a computer-readable storage medium for storing program code for executing the swing motion detection method based on a smart wearable device according to any embodiment of this invention.

[0124] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0125] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this invention are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0126] In the embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units through some interfaces, and may be electrical, mechanical, or other forms.

[0127] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0128] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0129] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0130] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A swing motion detection method based on a smart wearable device, characterized by, include: When a smart wearable device detects that a person has started to move, it collects motion data; The motion data is subjected to racket swing motion extraction based on dual analysis of velocity and electromyography signals to obtain a complete racket swing motion segment; Multidimensional features are extracted from the swing motion segment, and a multidimensional feature vector is constructed. The multidimensional feature vector and the swing motion fragment are input into a pre-trained ball motion recognition model to recognize ball swing motions, and the ball recognition result and swing type prediction result are output. 2.The smart wear device based swing motion detection method according to claim 1, wherein, The motion data includes acceleration and angular velocity data, as well as electromyography (EMG) signals; the extraction of the racket swing motion based on dual analysis of the motion data using velocity and EMG signals yields a complete racket swing motion segment, including: The acceleration and angular velocity data and the electromyographic signal are time-stamped and then filtered to obtain calibrated acceleration and angular velocity data and calibrated electromyographic signals. The time-domain integral and frequency-domain energy calculations are performed on the calibrated electromyographic signal to obtain the energy characteristics of the electromyographic signal; Based on the calibrated acceleration and angular velocity data and the electromyographic signal energy characteristics, a comprehensive swing judgment is made to determine the starting point and ending point of a complete swing motion. The acceleration time sequence and electromyography time sequence from the start point to the end point of the swing are extracted from the calibrated acceleration and angular velocity data and the calibrated electromyography signal to form a complete swing motion segment. 3.The smart wear device based swing motion detecting method according to claim 2, wherein, The smart wearable device has a built-in A+G sensor module and an electromyography (EMG) sensor; when the smart wearable device detects that the human body has started to move, it collects motion data, including: When an acceleration signal is detected by the A+G sensor module, it is determined that the human body has started to move. Using a time-series recording method, the A+G sensor module collects and records acceleration and angular velocity data, and the electromyography sensor collects and records the electromyographic signals generated during human muscle activity. 4.The smart wear device based swing motion detecting method according to claim 3, wherein, The method of comprehensively judging the swing based on the calibrated acceleration and angular velocity data and the electromyographic signal energy characteristics to determine the starting and ending points of a complete swing includes: Based on the calibrated acceleration and angular velocity data, the composite acceleration magnitude and composite angular velocity magnitude at each moment in the recorded time series are calculated one by one in chronological order. During the calculation process, if the combined acceleration modulus and the combined angular velocity modulus exceed the preset dynamic threshold and meet the correlation coefficient condition, and the electromyographic signal energy value is determined to be within the preset range based on the electromyographic signal energy characteristics, then the first time point is marked as the starting point of a complete swing motion. Continue the calculation. If there is a second time point when the combined acceleration modulus and the combined angular velocity modulus are lower than the preset dynamic threshold and can be maintained for a preset duration, and the electromyographic signal energy value is within the preset normal range based on the electromyographic signal energy characteristics, then mark the second time point as the swing end point corresponding to the swing start point. 5.The smart wear device based swing motion detecting method according to claim 1, wherein, The extracted multidimensional features include acceleration multidimensional features, angular velocity multidimensional features, and electromyographic signal energy features; The energy characteristics of the electromyographic signal include the time-domain integral value of the electromyographic signal and the energy proportion of the electromyographic signal in different frequency domains. 6.The smart wear device based swing motion detecting method according to claim 1, wherein, The pre-trained ball action recognition model includes a neural network model pre-trained based on ball recognition and several LSTM sub-models pre-trained based on swing action recognition; the step of inputting the multi-dimensional feature vector and the swing action fragment into the pre-trained ball action recognition model for ball swing action recognition, and outputting ball recognition results and swing type prediction results includes: The multidimensional feature vector is input into the neural network model to perform ball recognition and obtain the ball recognition result. Determine the target LSTM sub-model corresponding to the ball identification result, call the target LSTM sub-model to perform swing type identification on the swing action segment, and output the swing type prediction result. 7.The smart wear device based swing motion detection method according to any one of claims 1 to 6, characterized in that, The smart wearable device is communicatively connected to a smart terminal; the method further includes: Based on the calibrated acceleration and angular velocity data obtained after calibration and filtering, the peak swing velocity and peak swing acceleration are calculated respectively. Based on the peak swing acceleration, the hitting power index is estimated, and the hitting power index is corrected based on the electromyographic signal intensity to obtain the corrected hitting power index. The ball identification result, the swing type prediction result, the peak swing speed, the peak swing acceleration, the corrected hitting power index, and the current swing count are packaged and sent to the smart terminal.

8. A swing motion detecting apparatus based on a smart wearable device, characterized by, include: The data acquisition unit is used to collect motion data when the smart wearable device detects that the human body has started to move; The racket swing motion extraction unit is used to extract the racket swing motion based on the motion data through dual analysis of velocity and electromyography signals, so as to obtain a complete racket swing motion segment. A multidimensional feature extraction unit is used to extract multidimensional features from the swing motion segment and construct a multidimensional feature vector; The ball swing action recognition unit is used to input the multi-dimensional feature vector and the swing action segment into a pre-trained ball action recognition model to recognize the ball swing action, and output the ball recognition result and the swing type prediction result.

9. A smart wearable device, characterized by, The smart wearable device includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the swing motion detection method based on any one of claims 1-7 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program code for executing the racket swing action detection method based on a smart wearable device according to any one of claims 1-7.

Images