Real-time motion pattern switching recognition system and method based on single inertial sensor

Abstract

The application relates to the technical field of motion detection, in particular to a real-time motion mode switching identification system and method based on a single inertial sensor, which comprises a motion mode conversion data stream automatic acquisition module, a space-time gait parameter feature processing module and an improved K-medoids clustering algorithm model which are sequentially connected; the motion mode conversion data stream automatic acquisition module comprises an inertial motion sensing unit and a probability distribution modeling unit; the space-time gait parameter feature processing module comprises a feature extraction unit, a sorting unit and a cross-validation unit; and the improved K-medoids clustering algorithm model is used for classifying and training and predicting different switching motion states according to selected gait features, so as to realize accurate identification of a complex human motion switching process. The real-time motion mode switching identification system based on a single inertial sensor provided by the application mainly solves the problems of low accuracy and large complexity when a wearable device switches motion modes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of motion detection, in particular to a real-time motion mode switching recognition system and method based on a single inertial sensor. BACKGROUND

[0002] With the aggravation of population aging, motion recognition plays an important role in health monitoring, gait impairment assessment and wearable device control. There are different motion modes in the process of human lower limb movement, and the motion mode conversion such as walking-running, standing-up-sitting-down, going up and down stairs-walking on flat ground is the most basic gait instruction. However, neurodegenerative diseases such as Parkinson's disease can cause serious gait dysfunction. In the early stage of Parkinson's disease, the gait of patients mainly shows symptoms such as limb stiffness, unilateral limb weakness and reduced swing amplitude of both upper limbs, which can make the conversion between motion modes difficult and dangerous. Therefore, the detection of motion switching for people with movement disorders and the development of related systems have become an urgent need of the clinic and community.

[0003] There are mainly three solutions to the current motion mode switching problem: 1) threshold-based motion transition recognition, for example, using a potentiometer or other sensors connected to a rehabilitation device to monitor ankle and knee angles. However, this method is prone to false recognition because different thresholds need to be defined according to the anthropometry of each subjet; 2) different functional buttons are classified according to the activity transition phase, and the buttons are embedded in wearable devices that assist in motion switching support. Patients only need to press the trigger button to get external force assistance for motion state transition, but for patients, pressing the button itself is an additional challenge, and triggering the switch for many specific tasks will require many buttons; 3) wearable sensors such as inertial measurement units or electromyography sensors are used and machine learning methods are applied for activity recognition. However, real-time motion switching usually relies on sensors (such as accelerometers, gyroscopes, etc.) to capture motion data, and the accuracy, sampling rate and noise level of the sensors will affect the accuracy of recognition.

[0004] In summary, the current motion mode switching system generally relies on hardware trigger structure to achieve, which lacks feasibility for patients with severe motor disorders, or the threshold algorithm used is not good enough, with low accuracy. SUMMARY

[0005] The present application provides a real-time motion mode switching recognition system and method based on a single inertial sensor, which solves the problem of low accuracy and high complexity of wearable devices in motion mode switching.

[0006] To solve the above technical problems, the embodiment of the present application provides a real-time motion mode switching recognition system and method based on a single inertial sensor, comprising: a motion mode switching data stream automatic acquisition module, a space-time gait parameter feature processing module and an improved K-medoids clustering algorithm model connected in sequence; the motion mode switching data stream automatic acquisition module comprises an inertial motion sensing unit and a probability distribution modeling unit; wherein the inertial motion sensing unit is used to acquire nine-axis inertial data in a continuous time domain; the probability distribution modeling unit is used to perform segmentation processing on the nine-axis inertial data in the continuous time domain, identify the time domain interval in the switching motion state, and obtain the switching data stream; the space-time gait parameter feature processing module comprises a feature extraction unit, a sorting unit and a cross-validation unit; the feature extraction unit is used to extract different categories of time domain features in the switching data stream by using a sliding window; the sorting unit is used to sort the extracted different categories of time domain features; the cross-validation unit is used to cross-validate the sorted different categories of time domain features, train and verify the K-medoids switching recognition model, and obtain selected gait features; the improved K-medoids clustering algorithm model is used to classify and train different switching motion states according to the selected gait features, and realize accurate recognition of the complex human motion switching process.

[0007] In some example embodiments, the motion mode switching data stream automatic acquisition module further comprises: a power supply and a Bluetooth communication device matched with the inertial motion sensing unit; the inertial motion sensing unit acquires nine-axis inertial data in a continuous time domain at the start of motion, and transmits the acquired nine-axis inertial data in the continuous time domain to the upper computer through the Bluetooth communication device.

[0008] In some example embodiments, the inertial motion sensing unit is a waist inertial motion sensor.

[0009] In some example embodiments, the selected gait features include acceleration features and angular velocity features that can reflect the differences between different starting motion modes.

[0010] In some example embodiments, the sliding window adopts a sliding window with a size of 200 ms, an overlap degree of 95% and a step speed of 10 ms.

[0011] In some example embodiments, the different categories of time domain features include mean value, maximum value, minimum value, angular velocity signal amplitude area and acceleration average intensity.

[0012] In the second aspect, an embodiment of the present application also provides a real-time motion mode switching recognition method based on a single inertial sensor, comprising the following steps: first, based on the inertial motion sensing unit, nine-axis inertial data in the continuous time domain is obtained; and the nine-axis inertial data in the continuous time domain is segmented and processed to identify the time domain interval in the switching motion state to obtain a conversion data stream; then, a sliding window is used to extract different categories of time domain features in the conversion data stream, and the extracted time domain features of different categories are sorted and cross-validated, and the K-medoids switching recognition model is trained and verified to obtain selected gait features; finally, based on the selected gait features and the improved K-medoids clustering algorithm model, different switching motion states are classified, trained and predicted to achieve accurate recognition of complex human motion switching processes.

[0013] In some exemplary embodiments, a Bayesian update formula is used to segment the nine-axis inertial data in the continuous time domain. The Bayesian update formula is obtained by iteratively updating the posterior probability by multiplying the prior probability and the posterior probability. The Bayesian update formula is as follows:

[0014]

[0015] Among them, c n Represents a perception category; the perception category includes an activity state category and a transition stage category; z t represents the posterior probability of d at time t; z t-1 represents the posterior probability of the previous time t-1; P(c n |z t ) and P(z t |c n ) is the posterior probability at time t and hypothesis c n If z is observed when it is true t Likelihood; P(c n ) is the prior probability; P(z t |z t-1 ) is the marginal probability.

[0016] In some exemplary embodiments, sorting and cross-validating the extracted time-domain features of different categories includes: using a recursive feature elimination method to sort and cross-validate the extracted time-domain features of different categories.

[0017] In some example embodiments, based on the selected gait features and the improved K-medoids clustering algorithm model, different switching motion states are classified, trained and predicted, and the classification principle includes the following steps: step one, randomly selecting k data points as initial cluster centers for data initialization; wherein k represents the number of switching motion phase categories; step two, for each point in the data set, the Manhattan distance of each point to all cluster centers is calculated; the calculation formula of the Manhattan distance is shown as formula (2):

[0018]

[0019] In formula (2), x and y are the coordinates of two feature points in n-dimensional space:

[0020] Step three, according to the calculated Manhattan distance, each point is assigned to the nearest cluster center to form k clusters; step four, for each cluster, a new cluster center is found, so that the sum of the Manhattan distances of all points in the cluster to the new cluster center is minimized, thereby updating the cluster center; step five, repeat steps two to four until all center clusters no longer change, and the iteration is completed.

[0021] The technical scheme provided by the application has at least the following advantages:

[0022] The application provides a real-time motion mode switching recognition system and method based on a single inertial sensor, which comprises: a motion mode conversion data stream automatic acquisition module, a space-time gait parameter feature processing module and an improved K-medoids clustering algorithm model connected in sequence; the motion mode conversion data stream automatic acquisition module comprises an inertial motion sensing unit and a probability distribution modeling unit; wherein the inertial motion sensing unit is used to acquire nine-axis inertial data in a continuous time domain; the probability distribution modeling unit is used to segment and process the nine-axis inertial data in the continuous time domain, identify the time domain interval in the switching motion state, and obtain the conversion data stream; the space-time gait parameter feature processing module comprises a feature extraction unit, a sorting unit and a cross-validation unit; the feature extraction unit is used to extract different categories of time domain features in the conversion data stream by using a sliding window; the sorting unit is used to sort the extracted different categories of time domain features; the cross-validation unit is used to cross-validate the sorted different categories of time domain features, train and verify the K-medoids switching recognition model, and obtain selected gait features; the improved K-medoids clustering algorithm model is used to classify, train and predict different switching motion states according to the selected gait features, and realize accurate recognition of the complex human motion switching process.

[0023] In view of the shortcomings of data fusion of multiple sensors to realize recognition and cooperative control of switching motion, the application deploys a light-weight, low-power switching motion recognition system based on a single inertial motion sensor. The system mainly includes three parts: automatic acquisition of motion mode conversion data stream, spatiotemporal gait parameter feature processing, and improved design of K-medoids clustering algorithm. The application proposes a set of real-time motion mode switching control systems based on a single wearable inertial motion sensor unit, combines innovative methods such as probability distribution modeling and K-medoids clustering model, realizes automatic support switching between standing and sitting transition and other tasks related to gait, can significantly improve the switching motion recognition accuracy, and can be used for further development of clinical and community rehabilitation auxiliary intelligent equipment. BRIEF DESCRIPTION OF DRAWINGS

[0024] One or more embodiments are illustrated by way of example in the figures that form a part of this patent document, and in which like reference numbers typically indicate similar components in the claims, drawings and text in which:

[0025] Figure 1 FIG. 1 is a structural schematic diagram of a real-time motion mode switching recognition system based on a single inertial sensor according to an embodiment of the application.

[0026] Figure 2 FIG. 2 is a principle block diagram of a real-time motion mode switching recognition system based on a single inertial sensor according to an embodiment of the application.

[0027] Figure 3A FIG. 3 is an automatic cutting and acquisition of motion mode conversion data stream based on the Bayes formula according to an embodiment of the application. Figure 1 .

[0028] Figure 3B FIG. 4 is an automatic cutting and acquisition of motion mode conversion data stream based on the Bayes formula according to an embodiment of the application. Figure 2 .

[0029] Figure 4 FIG. 5 is a schematic diagram of a sliding window according to an embodiment of the application.

[0030] Figure 5 FIG. 6 is a schematic diagram of a training and test data preparation and recognition model generation workflow according to an embodiment of the application.

[0031] Figure 6 FIG. 7 is a structural schematic diagram of a real-time motion mode switching recognition system based on a single inertial sensor according to an embodiment of the application.

[0032] Figure 7is a clustering result schematic diagram of six center clusters formed by K-medoids in different switching motion mode stages provided by an embodiment of the present application.

[0033] Figure 8 is a real-predicted label confusion matrix result schematic diagram of different switching motion modes on a training set provided by an embodiment of the present application.

[0034] Figure 9 is a real-predicted label confusion matrix result schematic diagram of different switching motion modes on a test set provided by an embodiment of the present application.

[0035] Figure 10 is an algorithm comparison result schematic diagram provided by an embodiment of the present application. DETAILED DESCRIPTION

[0036] As known from the background, the current motion mode switching system generally relies on hardware trigger structure for implementation, and lacks feasibility for patients with severe movement disorders, or the threshold algorithm adopted is not good enough, and the accuracy is relatively low.

[0037] There are mainly the following three solutions to the current motion mode switching problem: 1) threshold-based motion transition recognition; 2) dividing different function buttons according to activity transition stage categories, and then embedding the buttons into a wearable device supporting auxiliary motion switching; 3) using wearable sensors such as inertial measurement units or electromyography sensors and applying machine learning methods for activity recognition. Specifically, a related technology discloses a motion mode switching recognition method based on K-Means and gait cycle similarity. First, a template vector corresponding to the category is generated offline by collecting data. When recognizing, the real-time collected data is processed and similarity analysis is performed with the template vector. According to the similarity result, the motion mode is classified, and the probability of the current result is given according to the similarity, solving the problem of large calculation amount or limited template matching precision in the prior art, and the feature vector set can be extracted semi-automatically. The real-time performance is good, and the precision is higher than that of the traditional template matching method. However, the K-Means algorithm is sensitive to outliers, and outliers may have a great influence on the clustering result, resulting in inaccurate transition motion mode recognition.

[0038] Another related technology discloses a wearable intelligent gait analyzer based on multi-sensor data fusion technology, comprising: a dual-channel surface electromyography sensor for collecting muscle contraction generated electrical signals; an inertial measurement unit for collecting angle, angular velocity and angular acceleration information of leg joints when the lower limbs of the human body move; a plantar film pressure sensor for collecting plantar pressure information; a main control board for fusing the collected data information through multi-sensor data fusion technology and D-S evidence theory fusion algorithm; a CS architecture network server for gait recognition through deep neural network intelligent algorithm. The invention balances the information intensity of different types of sensors by using multi-sensor data fusion technology, and improves the accuracy of action switching recognition by using deep neural network intelligent algorithm. However, the highly integrated design may increase the manufacturing and maintenance costs, and may also increase the complexity of the system, which may cause operation difficulty for non-professional users.

[0039] In the past, the start recognition of various movement patterns was mainly based on multiple types of sensors or threshold methods, and the following shortcomings and limitations existed in the development of the recognition algorithm:

[0040] 1) The traditional clustering algorithm such as k-means algorithm has a simple implementation, but since the cluster center is determined by the sample point mean, it may lead to the fact that the cluster center does not represent any actual switching movement state sample. This deviation may reduce the interpretability of the clustering result, making it difficult for users to understand the actual movement pattern behind the clustering result. In addition, sensor data may contain noise and outliers caused by other external factors. The k-means algorithm is sensitive to these outliers, which may cause the clustering result to deviate from the true transition movement pattern, thereby affecting the accuracy and reliability of the recognition.

[0041] 2) The threshold-based specific movement task switching solution is simple and easy to implement, for example, by monitoring the ankle and knee joint angle changes based on a threshold-based heuristic algorithm. However, due to the differences in the definition of anthropometry thresholds between different subjects, this method is susceptible to individual differences when dealing with large-scale samples, thereby increasing the risk of false recognition.

[0042] 3) Unlike traditional point-to-point connection structure data acquisition systems, complex scenarios such as rehabilitation tasks may require multiple types of sensors, which may have different sampling rates and communication protocols. In the case of simultaneous use of multiple data acquisition systems, asynchronous data may cause the time scale between different systems to drift, increasing the complexity of data integration and processing.

[0043] In view of the shortcomings of poor robustness and poor interpretability of the K-means algorithm, the K-medoids algorithm is creatively applied to motion mode switching recognition in the application, and the application provides a real-time motion mode switching recognition system and method based on a single inertial sensor, which comprises: a motion mode conversion data stream automatic acquisition module, a space-time gait parameter feature processing module and an improved K-medoids clustering algorithm model connected in sequence; the motion mode conversion data stream automatic acquisition module comprises an inertial motion sensing unit and a probability distribution modeling unit; wherein the inertial motion sensing unit is used to acquire nine-axis inertial data in a continuous time domain; the probability distribution modeling unit is used to segment the nine-axis inertial data in the continuous time domain, identify the time domain interval in the switching motion state, and obtain the conversion data stream; the space-time gait parameter feature processing module comprises a feature extraction unit, a sorting unit and a cross-validation unit; the feature extraction unit is used to extract different categories of time domain features in the conversion data stream by using a sliding window; the sorting unit is used to sort the extracted different categories of time domain features; the cross-validation unit is used to cross-validate the sorted different categories of time domain features, train and verify the K-medoids switching recognition model, and obtain selected gait features; the improved K-medoids clustering algorithm model is used to classify and train different switching motion states according to the selected gait features, and realize accurate recognition of the complex human motion switching process.

[0044] The application provides a real-time motion mode switching recognition system and method based on a single inertial sensor, which can automatically support switching between standing and sitting transitions and other tasks related to gait. The system is based on a single inertial sensor, which avoids the time scale drift error caused by cross-fusion of multiple sensor data, and selects the best classifier according to the cross-validation (Leave-One-Participant-Out, LOPOCV), recall rate and F1 score results, uses an improved K-medoids clustering algorithm, and can better improve the transition motion state recognition accuracy.

[0045] The embodiments of the application will be described in detail below with reference to the accompanying drawings. However, those skilled in the art can understand that in the embodiments of the application, many technical details are proposed in order to make the reader better understand the application. However, the technical solutions claimed by the application can be realized even without these technical details and various changes and modifications based on the following embodiments.

[0046] Reference Figure 1The embodiment of the application provides a real-time motion mode switching recognition system based on a single inertial sensor, which comprises: a motion mode conversion data stream automatic acquisition module 101, a space-time gait parameter feature processing module 102 and an improved K-medoids clustering algorithm model 103 connected in sequence; wherein the motion mode conversion data stream automatic acquisition module 101 comprises an inertial motion sensing unit 1011 and a probability distribution modeling unit 1012; wherein the inertial motion sensing unit 1011 is used for acquiring nine-axis inertial data in a continuous time domain; the probability distribution modeling unit 1012 is used for performing segmentation processing on the nine-axis inertial data in the continuous time domain, identifying a time domain interval in a switching motion state, and obtaining a conversion data stream; the space-time gait parameter feature processing module 102 comprises a feature extraction unit 1021, a sorting unit 1022 and a cross-validation unit 1023; the feature extraction unit 1021 is used for extracting different categories of time domain features in the conversion data stream by using a sliding window; the sorting unit 1022 is used for sorting the extracted different categories of time domain features; the cross-validation unit 1023 is used for cross-validating the sorted different categories of time domain features, training and verifying a K-medoids switching recognition model, and obtaining selected gait features; the improved K-medoids clustering algorithm model 103 is used for classifying and training different switching motion states according to the selected gait features, and realizing accurate recognition of a complex human motion switching process, so as to improve switching recognition accuracy and interpretability.

[0047] In view of the shortcomings of poor robustness and poor interpretability of the K-means algorithm, the application creatively applies the K-medoids algorithm to motion mode switching recognition, and proposes a set of real-time motion mode switching control systems based on a single wearable inertial motion sensing unit, which mainly includes motion mode conversion data stream automatic acquisition, space-time gait parameter feature processing and K-medoids clustering algorithm improvement design. Since the center point (medoids) of the K-medoids is an actual sample point selected from the data set, instead of the mean value of the sample point as in the K-means, the sensitivity to outliers is smaller. In the K-medoids algorithm, each cluster can be regarded as a specific motion mode switching waveform, and the center point is a typical representative of the mode. This helps researchers and developers better understand the data and the identified mode. In motion switching recognition, due to the error of the sensor or the interference of the environment, there may be noise in the data, and the K-medoids can effectively reduce the influence of the noise on the clustering result.

[0048] In addition, the data fusion of multiple sensors has the shortcomings of complex recognition and cooperative control complexity. The application is based on a single inertial motion sensor, and a lightweight and low-power switching motion recognition system is deployed. Since the single IMU is simple in design and has relatively low computing requirements, it is easy to integrate into clothes, bracelets and other devices, and will not have too much impact on the user's daily life. At the same time, the single inertial sensor can process data in real time to provide immediate motion mode feedback to the user.

[0049] Therefore, the application provides a real-time motion mode switching recognition system and method based on a single inertial sensor. The system is a real-time motion mode switching recognition control system that automatically supports switching between standing and sitting transitions and other tasks related to gait. The scheme is based on a single inertial sensor, which avoids time scale drift errors caused by cross-fusion of multiple sensor data. At the same time, according to the Leave-One-Participant-Out cross-validation (LOPOCV), recall rate and F1 score results, the best classifier is selected, and the improved K-medoids clustering algorithm can better improve the transition motion state recognition accuracy.

[0050] It should be noted that in the motion mode transition data stream automatic acquisition module 101, the nine-axis inertial data in the continuous time domain is segmented by the Bayesian update formula to obtain the transition data stream; in the spatiotemporal gait parameter feature processing module 102, different categories of time domain features are extracted from the transition data stream using a sliding window, and the extracted different categories of time domain features are sorted and cross-validated to train and verify the K-medoids switching recognition model; it should be noted that the K-medoids switching recognition model is a switching recognition model based on the K-medoids clustering algorithm, and the K-medoids switching recognition model is trained and verified to obtain the improved K-medoids clustering algorithm model 103. Through the improved K-medoids clustering algorithm model 103, combined with the selected gait features, different switching motion states are classified, trained and predicted to realize accurate recognition of the complex human motion switching process, thereby improving the switching recognition accuracy and interpretability.

[0051] In some embodiments, the motion mode transition data stream automatic acquisition module further comprises a power supply and a Bluetooth communication device matched with the inertial motion sensing unit; the inertial motion sensing unit acquires nine-axis inertial data in a continuous time domain at the start of motion, and transmits the acquired nine-axis inertial data in the continuous time domain to the upper computer through the Bluetooth communication device.

[0052] The principle diagram of the real-time starting motion recognition and motion mode switching system proposed in the application is as follows: Figure 2The system is composed of a hardware structure and an algorithm model. The core hardware structure includes a waist inertial motion sensor, a matching power supply and a Bluetooth communication device. The inertial motion sensor starts to acquire nine-axis inertial data in a continuous time domain at the start of motion, and transmits the acquired nine-axis inertial data in the continuous time domain to an upper computer through the Bluetooth communication device. Then, data preprocessing is performed in the calculation module to obtain derived parameters such as pitch angle, heading angle and roll angle. Subsequently, on the server side, the inertial sensing data collected in real time is subjected to downsampling and filtering processing, and conversion data stream (also referred to as switching data stream) is obtained according to the Bayes update formula. Then, different categories of time domain features are extracted in the conversion data stream by using a sliding window, and then the K-medoids switching recognition model is trained and verified for testing, so as to obtain selected gait features, i.e. acceleration features and angular velocity features that can reflect the differences between different starting motion modes. Then, the selected gait features are combined with the improved K-medoids clustering algorithm model of the application, and based on the classification principle of the K-medoids clustering algorithm, different switching motion states are classified, trained and predicted to realize accurate recognition of the complex human motion switching process.

[0053] In some embodiments, the inertial motion sensing unit is a waist inertial motion sensor.

[0054] In some embodiments, the selected gait features include acceleration features and angular velocity features that can reflect the differences between different starting motion modes, such as acceleration average intensity and angular velocity signal amplitude area.

[0055] In some embodiments, the sliding window adopts a sliding window with a size of 200 ms, an overlap degree of 95% and a step speed of 10 ms.

[0056] In some embodiments, the different categories of time domain features include average value, maximum value, minimum value, angular velocity signal amplitude area and acceleration average intensity.

[0057] In some embodiments, the Bayes update formula is used to segment the nine-axis inertial data in the continuous time domain. The Bayes update formula is obtained by iteratively updating the posterior probability through the product of the prior probability and the posterior probability. The Bayes update formula is as follows:

[0058]

[0059] wherein c n represents a perception category; the perception category includes an activity state category and a conversion phase category; z t represents the d posterior probability of time t; z t-1 represents the posterior probability of the previous time t-1 of time t; P(c n |z t) and P(z t |c n ) is the posterior probability at time t and hypothesis c n If true, we observe z t Likelihood; P(c n ) is the prior probability; P(z t |z t-1 ) is the marginal probability.

[0060] For the automatic collection of motion mode conversion data stream, such as Figure 3A and Figure 3B As shown, for the acceleration data obtained by the inertial motion sensor unit, this application proposes a belief network to model the probability distribution of temporal reasoning, and can automatically segment the perception category data through the Bayesian formula and sequence analysis method. The perception category data includes: activity state category data (sitting, standing) and transition stage category data (sitting-standing, standing-sitting).

[0061] Bayesian Update: The Bayesian formula iteratively updates the posterior probability based on the product of the prior probability and the posterior probability. The measurement value z represents the acceleration signal emitted by the wearable sensor on the participant's thigh. The activity state category and transition stage category are represented by c. n Each category c n By (u k , v l ) definition, where u k where k = 1, 2, ..., K; v l Where l = 1, 2, ..., L, respectively represents the active state and the transition stage, wherein the Bayesian update formula is shown in formula (1).

[0062] Among them, in formula (1), P(c n |z t ) and P(z t |c n ) is the posterior probability at time t and hypothesis c n If true, we observe z t The likelihood of . Prior probability P(c n ) uses an initial uniform distribution, and then updates it with the posterior probability obtained from the previous t-1 as time goes by. The marginal probability P(z t |z t-1 ) is used to ensure that the probability is between 0 and 1. Among them, the distribution P(c n |z t-1 ) is a priori, as mentioned above, for the iteration time t = 0, the uniform distribution P(c n )=P(c n|z0)=1 / N, for time t>0, the prior adopts the estimated posterior distribution from the previous iteration t-1. The marginal posterior of the active state and transition phase: the perception category c n The posterior probability of the activity state u is sitting-standing and standing-sitting activities k and transition phase v l The belief of a single activity state and transition phase is given by the marginal posterior probability, as shown in formula (2) and formula (3):

[0063]

[0064]

[0065] The activity state category P(u k |z t ) by the joint distribution P(u k , v l ) is obtained by summing. Similarly, the transition stage class P(v1|z t ) by performing P(u k , v l ) is obtained by summing. Stopping rule and decision making: Once the confidence threshold β threshold is exceeded, the accumulation of evidence or sensor measurements performed by the Bayesian formula stops. This event triggers the decision process to estimate the perception category of the current activity state and transition phase using maximum a posteriori (MAP) estimation as follows:

[0066] If P(v k |z t )>β threshold ,So:

[0067]

[0068] If P(v l |z t )>β threshold ,So:

[0069]

[0070] in, The activity state is estimated by and transition phase The estimated class of the composition. Confidence threshold βthreshold = [0.0, 0.5, ..., 0.99] adjusts the confidence level of the transition data stream acquisition method to achieve the desired decision and segmentation accuracy. In addition, the parameter βthreshold allows controlling the trade-off between accuracy and reaction time in distinguishing between active state and transition phase data.

[0071] In the spatio-temporal gait parameter feature processing part, as shown in FIG. 3, first, in order to find the best window size for feature extraction, the time of each subject performing different activities was calculated, and the shortest execution time of all participants was determined. Considering that the shortest activity time of some participants occurs within 200 milliseconds, the selected window size should not be less than 200 milliseconds, so as not to miss any activities, and will not have a large overlap with other activities. Therefore, the final window size is set to 200 ms, and the overlap rate is 95%, which is used to process the transition between different activities and shorten the classification delay in real time.

[0072] For each sliding window, 5 types of time domain features were extracted, including mean, maximum, minimum, angular velocity signal amplitude area, and acceleration average intensity, etc. Then, the extracted features were sorted and cross-validated by the recursive feature elimination method. By using the support vector machine (SVM) estimator to delete 0-21 features to select a feature subset, the different types of time domain features extracted were sorted and cross-validated by the recursive feature elimination method. Then, 4 features were selected according to the cross-validation score of the model (the best subset of features). These are the 4 features used for all the switch recognition models. Before training the switch recognition model, the selected features were scaled using the min-max scaling. At the same time, the signals were manually labeled using the video captured synchronously. Each moving window was labeled using the majority voting criterion, which means that if a task is the main task in the time window, the window will be labeled using the label of the dominant task. Each labeled window was combined with the generated feature vector to construct a training vector. In addition, in order to have a balanced number of samples between tasks in the training dataset, the tasks with a larger number of samples were down-sampled.

[0073] In the inertial motion data obtained from 10 healthy adults, leave-one-out cross-validation (LOPOCV) was performed for each classification method. In this way, each switch recognition model was trained and tested 10 times. Iteratively, the data of 9 participants were used as the training dataset for training the model, and the data of 1 participant were used to test and evaluate the performance (the data of each participant was used 9 times in the training dataset and 1 time in the test dataset). The average of the F1 score was reported to compare the performance.

[0074] As Figure 5As shown, the motion data is randomly divided into training data (training data set) and test data (test data set): 80% and 20% respectively. The random selection method ensures equal representation of all activities in the training and test data sets. The training data is used to develop the final K-medoids switch recognition model. The last participant is later used in a real-time scenario to check the performance of the final K-medoids switch recognition model. The data preparation process for the training and test data is the same: from signal preprocessing, then labeling and feature extraction. The features extracted from the training data and the corresponding labels for each feature set are used in the learning module for the switch recognition model generation. Therefore, the generated K-medoids switch recognition model is checked using the features extracted from the test data set in the inference module. Finally, the predicted labels are compared with the actual labels to check the performance of the inference module.

[0075] As shown in Figure 6 The embodiments of the present application also provide a real-time motion mode switching recognition method based on a single inertial sensor, comprising the following steps:

[0076] Step S1, based on the inertial motion sensing unit, nine-axis inertial data in a continuous time domain is obtained; and the nine-axis inertial data in the continuous time domain is segmented and processed to identify the time domain interval in the switching motion state, and a conversion data stream is obtained.

[0077] Step S2, different categories of time domain features are extracted from the conversion data stream using a sliding window, and the extracted different categories of time domain features are sorted and cross-validated, and a K-medoids switch recognition model is trained and verified, and selected gait features are obtained.

[0078] Step S3, based on the selected gait features and the improved K-medoids clustering algorithm model, different switching motion states are classified, trained and predicted, and the accurate recognition of the complex human motion switching process is realized.

[0079] In some embodiments, the sorting and cross-validation of the extracted different categories of time domain features in step S2 include: using a recursive feature elimination method to sort and cross-validate the extracted different categories of time domain features.

[0080] The K-medoids algorithm allows the Manhattan distance to be used to calculate the similarity of each point to the center of the cluster to which it belongs. Since the Manhattan distance has good robustness to noise and outliers in the data, and the calculation of distance only involves addition and subtraction, the calculation speed is faster than the Euclidean distance. In motion recognition, this can provide more flexibility to adapt to different switching motion modes and ensure the real-time performance of the motion switching recognition process.

[0081] In some embodiments, the selected gait features and the improved K-medoids clustering algorithm model in step S3 are used to classify and predict different switching motion states, and the classification principle includes the following steps: step one, randomly select k data points as initial cluster centers to initialize the data; wherein k represents the number of switching motion phase categories; step two, for each point in the data set, calculate the Manhattan distance of each point to all cluster centers; the calculation formula of the Manhattan distance is shown as formula (2):

[0082]

[0083] In formula (2), x and y are the coordinates of two feature points in n-dimensional space:

[0084] Step three, according to the calculated Manhattan distance, each point is assigned to the nearest cluster center to form k clusters; step four, for each cluster, find a new cluster center so that the sum of the Manhattan distances of all points in the cluster to the new cluster center is minimized, thereby updating the cluster center; step five, repeat steps two to four until all center clusters no longer change, and the iteration is completed.

[0085] The application proposes a real-time motion mode switching control system based on a single wearable inertial motion sensor unit, models the probability distribution of temporal reasoning through a belief network, and automatically segments activity state category data (sitting, standing) and transition phase category data (sitting- standing, standing-sitting) through the Bayesian formula and sequence analysis method. Moreover, the application uses a sliding window with a size of 200 ms and a step speed of 10 ms to collect features, and proposes angular velocity signal amplitude area and acceleration average intensity as difference feature derivation parameters of switching motion states. At the same time, the application uses an improved k-medoids algorithm to increase the real-time performance of motion switching recognition.

[0086] Compared with the prior art, the real-time motion mode switching recognition system and method based on a single inertial sensor provided by the application has the following advantages: on the one hand, the motion mode switching control system proposed by the application applies the K-medoids algorithm to motion switching recognition, which can effectively reduce the influence of noise generated by sensor errors or environmental interference on clustering results. On the other hand, in view of the complexity of the recognition and collaborative control of the motion switching of multiple sensors, the application deploys a lightweight, low-power switching motion recognition system based on a single inertial motion sensor. In addition, the application performs automatic acquisition of transition data streams and specific data feature extraction through a sliding window based on the Bayesian update formula, and the system realizes automatic support switching between standing and sitting transitions and other gait-related tasks, which is helpful for the further development of rehabilitation auxiliary intelligent devices.

[0087] Figure 7 Six center clusters formed by K-medoids are shown for different switching motion pattern stages; Figure 8 Real-predicted label confusion matrix of different switching motion patterns on the training set is shown; Figure 9 Real-predicted label confusion matrix of different switching motion patterns on the test set is shown. The system of the present application has been tested and verified, and the results prove that the proposed real-time motion pattern switching system has an average accuracy of more than 90.2% in identifying different switching motion states offline. The clustering results obtained by inputting the conversion data stream into the K-medoids model through the Bayesian update formula are shown in Figure 7 In addition, according to Figure 8 The confusion matrix classification effect on the training set and Figure 9 The test set, it can be obtained that the clustering recognition results on the training set are all 100%, while the prediction accuracy of the six switching motion patterns "walking-running", "running-walking", "going upstairs-going downstairs", "going downstairs-going upstairs", "sitting-standing up", and "standing-sitting down" on the test set is 100%, 77.8%, 86.4%, 94.1%, 84.6%, and 98.1%, respectively.

[0088] Figure 10 The box plot comparison analysis diagram of the improved K-medoids clustering algorithm of the present application and other clustering algorithms is shown. From the box plot distribution of the algorithm recognition accuracy Figure 10 It can be concluded that the improved medoids clustering algorithm has the best classification effect among the above algorithms.

[0089] The present application provides a real-time motion pattern switching recognition system and method based on a single inertial sensor, which solves the problem of low accuracy and large complexity of wearable devices when switching motion patterns. The system can be applied to patients with Parkinson's disease, Alzheimer's disease, multiple sclerosis and other neurodegenerative diseases, and realizes automatic transition switching between standing and sitting and other gait-related tasks based on an inertial sensor unit. The system combines innovative methods such as probability distribution modeling and K-medoids clustering model, which can significantly improve the switching motion recognition accuracy and can be used for further development of clinical and community rehabilitation auxiliary intelligent devices.

[0090] According to the technical scheme, the application provides a real-time motion mode switching recognition system and method based on a single inertial sensor, which comprises a motion mode conversion data stream automatic acquisition module, a time-space gait parameter feature processing module and an improved K-medoids clustering algorithm model connected in sequence. The motion mode conversion data stream automatic acquisition module comprises an inertial motion sensing unit and a probability distribution modeling unit. The inertial motion sensing unit is used to acquire nine-axis inertial data in a continuous time domain. The probability distribution modeling unit is used to perform segmentation processing on the nine-axis inertial data in the continuous time domain, recognize the time domain interval in the switching motion state, and obtain the conversion data stream. The time-space gait parameter feature processing module comprises a feature extraction unit, a sorting unit and a cross-validation unit. The feature extraction unit is used to extract different categories of time domain features in the conversion data stream by using a sliding window. The sorting unit is used to sort the extracted different categories of time domain features. The cross-validation unit is used to cross-validate the sorted different categories of time domain features, train and verify the K-medoids switching recognition model, and obtain selected gait features. The improved K-medoids clustering algorithm model is used to classify and train different switching motion states according to the selected gait features, and realize accurate recognition of the complex human motion switching process.

[0091] In view of the shortcomings that the data fusion of multiple sensors is used to realize the recognition and collaborative control of motion switching and the complexity is large, the application deploys a light-weight and low-power switching motion recognition system based on a single inertial motion sensor. The system mainly comprises three parts: motion mode conversion data stream automatic acquisition, time-space gait parameter feature processing and K-medoids clustering algorithm improvement design. The application proposes a set of real-time motion mode switching control system based on a single wearable inertial motion sensing unit, combines innovative methods such as probability distribution modeling and K-medoids clustering model, realizes automatic support switching between standing and sitting transition and other tasks related to gait, can significantly improve the switching motion recognition accuracy, and can be used for further development of clinical and community rehabilitation auxiliary intelligent equipment.

[0092] Those skilled in the art can understand that the above-mentioned embodiments are specific examples for implementing the application, and in actual application, various changes can be made in form and details without departing from the spirit and scope of the application. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the application, therefore the protection scope of the application should be limited by the scope defined in the claims.

Claims

1. A real-time motion pattern switching recognition system based on a single inertial sensor, characterized in that, The application relates to a motion mode conversion data stream automatic acquisition module, a space-time gait parameter feature processing module and an improved K-medoids clustering algorithm model connected in sequence. The motion mode conversion data stream automatic acquisition module comprises an inertial motion sensing unit and a probability distribution modeling unit; the inertial motion sensing unit is used for acquiring nine-axis inertial data in a continuous time domain; the probability distribution modeling unit is used for segmenting the nine-axis inertial data in the continuous time domain, identifying time domain intervals in a switching motion state and obtaining conversion data streams. The space-time gait parameter feature processing module comprises a feature extraction unit, a sorting unit and a cross-validation unit; the feature extraction unit is used for extracting different types of time domain features in the conversion data streams by using a sliding window; the sorting unit is used for sorting the extracted different types of time domain features; and the cross-validation unit is used for cross-validating the sorted different types of time domain features, training and verifying a K-medoids switching identification model and obtaining selected gait features. The improved K-medoids clustering algorithm model is used for classifying and training different switching motion states according to the selected gait features and realizing accurate identification of a complex human motion switching process. The nine-axis inertial data in the continuous time domain are segmented by using a Bayesian updating formula. The Bayesian updating formula is obtained by iteratively updating a posterior probability by a product of a prior probability and the posterior probability; and the Bayesian updating formula is as follows: The motion mode conversion data stream automatic acquisition module further comprises a power supply and a Bluetooth communication device matched with the inertial motion sensing unit. （1） wherein c n represents a perception class; the perception class includes an activity state class and a transition phase class; z t represents a d posterior probability of time t; z t-1 represents a posterior probability of time t-1 of the previous time t-1; P(c n |z t ) and P(z t |c n ) are likelihoods of observing z t in the case where the hypothesis c n is true; P(c t ) is a prior probability; and P(z t-1 |z n ) is a marginal probability.

2. The real-time motion pattern switching recognition system based on a single inertial sensor according to claim 1, characterized in that, The inertial motion sensing unit acquires the nine-axis inertial data in the continuous time domain at the start of motion and transmits the acquired nine-axis inertial data in the continuous time domain to an upper computer through the Bluetooth communication device. The inertial motion sensing unit is a waist inertial motion sensor. 3.The single inertial sensor based real-time motion pattern switching recognition system according to claim 1, wherein, The selected gait features comprise acceleration features and angular velocity features capable of reflecting differences of different starting motion modes.

4. The single inertial sensor based real-time motion pattern switching recognition system according to claim 1, characterized in that, The sliding window adopts a sliding window with a size of 200 ms, an overlapping degree of 95% and a step speed of 10 ms.

5. The single inertial sensor based real-time motion pattern switching recognition system according to claim 1, characterized in that, The different types of time domain features comprise an average value, a maximum value, a minimum value, an angular velocity signal amplitude area and an acceleration average intensity.

6. The single inertial sensor based real-time motion pattern switching recognition system according to claim 1, characterized in that, The application further relates to a method for identifying a complex human motion switching process.

7. A real-time motion pattern switching recognition method based on a single inertial sensor, applied to the real-time motion pattern switching recognition system based on a single inertial sensor according to any one of claims 1 to 6, characterized in that, Nine-axis inertial data in a continuous time domain are acquired based on an inertial motion sensing unit; the nine-axis inertial data in the continuous time domain are segmented, time domain intervals in a switching motion state are identified and conversion data streams are obtained; Different types of time domain features are extracted in the conversion data streams by using a sliding window, the extracted different types of time domain features are sorted and cross-validated, a K-medoids switching identification model is trained and verified, and selected gait features are obtained; Different switching motion states are classified and trained based on the selected gait features and an improved K-medoids clustering algorithm model, and accurate identification of a complex human motion switching process is realized. ​ The Bayesian updating formula is used to segment nine-axis inertial data in continuous time domain; The Bayesian updating formula is obtained by iterative updating of the posterior probability through the product of the prior probability and the posterior probability; the Bayesian updating formula is as shown below: （1） where c n represents a perception class; the perception class includes an activity state class and a transition phase class; z t represents a d posterior probability of time t; z t-1 represents a posterior probability of time t-1 of the previous time t-1; P(c n |z t ) and P(z t |c n ) are a likelihood of observing z n in the case where the hypothesis c t is true; P(c n ) is a prior probability; and P(z t |z t-1 ) is a marginal probability. 8.The single-inertial-sensor-based real-time motion pattern switching recognition method according to claim 7, wherein, The extracted time domain features of different categories are sorted and cross-validated, including: The recursive feature elimination method is used to sort and cross-validate the extracted time domain features of different categories. 9.The single-inertial-sensor-based real-time motion pattern switching recognition method according to claim 7, wherein, Based on the selected gait features and the improved K-medoids clustering algorithm model, different switching motion states are classified, trained and predicted, and the classification principle includes the following steps: Step one, randomly select k points as initial cluster centers to initialize the data; wherein k represents the number of switching motion phase categories; Step two, calculate the Manhattan distance of each feature point to all cluster centers; the calculation formula of the Manhattan distance is as shown in formula (2): （2） In formula (2), x and y are the coordinates of the feature point in n-dimensional space: Step three, according to the calculated Manhattan distance, each feature point is assigned to the nearest cluster center to form k clusters; Step four, for each cluster, find a new cluster center so that the sum of the Manhattan distances of all feature points in the cluster to the new cluster center is minimum, thereby updating the cluster center; Step five, repeat steps two to four until all cluster centers no longer change, and the iteration is completed.

Images