Upper limb multi-joint motion intention recognition method based on electromyographic signals
By combining time-frequency analysis of electromyographic signals and sub-model training with the Viterbi decoding algorithm, the problems of data dependence and prediction instability in traditional holistic modeling strategies are solved, achieving efficient and accurate recognition of multi-joint movements of the upper limb, which is suitable for applications such as prosthetic control and rehabilitation training.
Patent Information
- Application Number
- CN202511086440.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-04
- Publication Date
- 2025-12-16
AI Technical Summary
Traditional holistic modeling strategies suffer from problems such as high data acquisition costs, poor model generalization ability, poor structural extensibility, and unstable time-series prediction in upper limb multi-joint motion recognition, making them difficult to adapt to the recognition of complex action combinations in dynamic scenes.
An upper limb multi-joint movement intention recognition method based on electromyography (EMG) signals is adopted. By acquiring and preprocessing EMG signals, sub-models of the shoulder and elbow joints are trained respectively. The Viterbi decoding algorithm is used for temporal smoothing optimization to achieve accurate recognition and stable prediction of movements.
It improves the accuracy and stability of motion recognition, reduces dependence on training data, has good scalability and real-time application capabilities, and is suitable for scenarios such as prosthetic control and rehabilitation training.
Smart Images
Figure CN121129291A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of electromyography (EMG) signal processing and recognition technology, and in particular to a method for recognizing the intention to move multiple joints of the upper limb based on EMG signals. Background Technology
[0002] With the rapid development of fields such as human-computer interaction, intelligent prostheses, and rehabilitation assistive systems, bioelectrical signals are generated before limb movement. By collecting and decoding bioelectrical signals, we can discover the relationship between signals and movement, which is a very useful method for predicting human limb movement trends.
[0003] Traditional action recognition methods often employ a holistic modeling strategy, treating multiple joints and movements of the upper limbs as a unified whole and classifying them using end-to-end machine learning or deep learning models. While these methods achieve some success in single-action recognition, they still face the following key challenges in recognizing complex combinations of actions:
[0004] 1. The overall modeling method is highly dependent on training samples. When the number of action categories increases or new combinations of actions appear, the system must collect a large amount of new training data, which significantly increases the data acquisition cost and training complexity, and the model has poor generalization ability.
[0005] 2. Lack of structural scalability. When using a holistic model to recognize upper limb combined movements, the introduction of new combined movements often means that the model structure and parameters need to be completely retrained, leading to difficulties in system maintenance and upgrades. This is not conducive to deployment in dynamic scenarios, such as customizing the movement library for prosthetic users and updating movements during rehabilitation training phases.
[0006] 3. Traditional methods struggle to guarantee the stability of temporal prediction. Since most action recognition methods employ frame-by-frame or window-by-window prediction strategies, they cannot model the continuity and contextual relationships of actions in the temporal dimension. This can easily lead to problems such as prediction jumps and jitter, affecting the stability and availability of the system in real-time applications. Summary of the Invention
[0007] To address the above technical problems, this invention provides a method for recognizing upper limb multi-joint movement intentions based on electromyographic signals, comprising the following steps:
[0008] S1. Acquire electromyographic signals of single joint movements of the upper limb and preprocess them. Analyze the time-frequency energy changes of the signals to automatically detect the active segment.
[0009] S2. Extract the temporal features of electromyography signals, train the shoulder joint sub-model and elbow joint sub-model respectively, and output the probability distribution of their respective movements.
[0010] S3. After concatenating the output probability vectors of the shoulder joint sub-model and the elbow joint sub-model, input them into the fusion unit to predict the combined action.
[0011] S4. Perform temporal smoothing optimization on the predicted sequence using the Viterbi decoding algorithm.
[0012] The technical solution further defined in this invention is:
[0013] Furthermore, step S1 specifically includes the following sub-steps:
[0014] S1.1. Attach electromyography (EMG) sensors to the muscle groups involved in shoulder and elbow joint movements to acquire EMG signals.
[0015] S1.2. The collected electromyographic signals are preprocessed. A fourth-order Butterworth bandpass filter of 20-400Hz is used to filter out high-frequency noise above 400Hz, and a 50Hz notch filter is used to filter out power line interference. The signal envelope is obtained by passing through a full-wave rectifier and a 1Hz first-order Butterworth low-pass filter.
[0016] S1.3 Automatically detect the active segment by the time-frequency energy change of the signal, and finally automatically identify the electromyographic activation segment and generate a label sequence with the same duration as the original signal.
[0017] As described above, in the method for recognizing upper limb multi-joint movement intention based on electromyography (EMG) signals, in step S1.1, the EMG signals are collected by an EMG signal sensor and amplified by a signal amplification circuit; the EMG signal sensor is placed at the center of the muscle and parallel to the direction of the muscle fibers, and the original signal is collected at a sampling frequency of 2000Hz.
[0018] As described above, the upper limb multi-joint movement intention recognition method based on electromyography signals includes the following sub-steps in step S1.3:
[0019] S1.3.1 Perform continuous wavelet transform on the filtered electromyographic signal to obtain the time-frequency signal, and calculate the average value of the time-frequency energy in the active frequency band of 20-150Hz in the frequency dimension to form a time-series energy curve;
[0020] S1.3.2 Acquire an electromyographic signal in a resting state, perform the previous step to obtain the energy curve in the resting state, and calculate its mean and standard deviation.
[0021] S1.3.3 Set the activation detection threshold as: Threshold = mean + k·std, where k is an empirical coefficient;
[0022] S1.3.4 Automatically identify electromyographic activation segments and generate a label sequence with the same duration as the original signal.
[0023] As described above, in the method for recognizing upper limb multi-joint movement intention based on electromyography (EMG) signals, in step S2, the temporal features of the automatically segmented EMG signals are extracted through a sliding window, namely root mean square value, mean absolute value, wavelength, integral EMG, and slope change; the EMG signal features of the muscle groups involved in shoulder and elbow joint movements are input into the corresponding shoulder joint sub-model and elbow joint sub-model, respectively.
[0024] As described above, the temporal characteristics of the automatically segmented electromyographic (EMG) signal for upper limb multi-joint movement intention recognition are shown in the following formula:
[0025]
[0026] Where, x i Let x represent the signal amplitude at the i-th sampling point, N represent the number of signal points within the sliding window, and x represent the signal amplitude at the ith sampling point. th This represents the noise suppression threshold.
[0027] As described above, in the method for recognizing upper limb multi-joint movement intention based on electromyography signals, in step S2, the movement forms of the shoulder include flexion, extension, adduction, and abduction of the shoulder joint, and the movement forms of the elbow include flexion and extension of the elbow joint. Both the shoulder joint sub-model and the elbow joint sub-model use XGBoost as the classifier. After training, the output of each sub-model is a probability distribution normalized by softmax, which is used to reflect the model's confidence level for various types of movements.
[0028] As described above, in the method for recognizing upper limb multi-joint movement intention based on electromyography signals, in step S3, the fusion unit adopts a multilayer perceptron model, which concatenates the probability distributions of each single joint movement output by the shoulder joint sub-model and the elbow joint sub-model into a probability vector and inputs it into the fusion unit for training. After training, it is used to predict multi-joint combined movements.
[0029] As described above, an upper limb multi-joint movement intention recognition method based on electromyography signals is used. The fusion unit is trained through supervised learning, with the goal of minimizing the cross-entropy loss function between the output probability distribution and the real combined action label.
[0030] The fusion layer consists of an input layer, a first hidden layer, a second hidden layer, and an output layer. The input layer has 6 nodes, which extract the probability output vectors of the two shoulder joint sub-models and the elbow joint sub-model, and concatenate them frame by frame in time to form a joint probability vector P. joint This vector is output by the shoulder model. Elbow model output It is pieced together, that is The first hidden layer consists of 32 neurons and a ReLU activation function; the second hidden layer consists of 16 neurons and a ReLU activation function; the output layer consists of 5 nodes and a softmax activation function. The 5 nodes correspond to 5 types of combined actions, including picking up objects from a height, drinking water from a cup, pushing a door, lifting objects, and swinging an arm.
[0031] As described above, the upper limb multi-joint movement intention recognition method based on electromyography signals includes the following sub-steps in step S4:
[0032] S4.1 After the fusion unit predicts the continuous sliding time windows, it obtains an output matrix over a time series, denoted as:
[0033] O = [o1 o2 … o] T ]
[0034] Among them, o t This represents the combined action result output by the fusion unit in frame t;
[0035] S4.2 Construct a finite-state Markov model to model the transition probability relationships between combined actions, where the state set represents all combined action categories (e.g., S = {s1 ... s2}). T});
[0036] S4.3 Construct the state transition probability matrix, which is defined as the state s i Transition to state s j The probability of the action is denoted as A(i,j), which is obtained by statistically analyzing the frequency of action labels in adjacent time windows in the training set.
[0037] S4.4. Using the fusion output sequence O as the input observation sequence, and considering it as the observation result produced in a certain state, apply the Viterbi algorithm to perform optimal path decoding and calculate the state path with the highest probability:
[0038] S * =argmax P(S|O)
[0039] Where P(S|O) represents the probability of the state sequence S occurring given the observation sequence O, and the process of finding the state path S with the highest probability step by step. * .
[0040] The beneficial effects of this invention are:
[0041] (1) In this invention, the electromyography signal is automatically activated by time-frequency energy analysis, which can accurately identify the start and end times of muscle activation, and facilitate the segmentation of electromyography signals in different states for subsequent modeling. This method is based on the characteristic that the energy spectrum distribution of electromyography signals in the activated state is significantly higher than that in the resting state, thus avoiding the problem of insufficient adaptability caused by manually setting a fixed muscle activation threshold. In order to determine the time range of the action, labels are added to the training data. Traditional methods rely on human observation of waveforms or the use of sensors (such as IMU), which have problems such as strong subjectivity, low efficiency and large deviation. This method can be used for automatic labeling of data, improve the consistency and accuracy of labels, and provide accurate action interval data for subsequent feature extraction and model training.
[0042] (2) In this invention, the shoulder joint and elbow joint sub-models are trained in parallel in the model structure design, which can effectively realize the "decoupled recognition" of upper limb movements. Compared with using a unified model to recognize all movements directly, this structure can model the electromyographic response features of different joints separately. Each sub-model only needs to focus on the feature differences of one joint, avoiding interference caused by the cross-action of multiple joints, thereby improving the accuracy of single joint recognition. In the output stage, the probability distributions of the two sub-models are spliced and input into the fusion module, which can realize the automatic construction and prediction of combined movements, and has good scalability and interpretability.
[0043] (3) In this invention, the Viteibi smoothing algorithm is introduced in the post-processing stage of the action prediction sequence, which improves the stability and continuity of the overall recognition system from the perspective of temporal modeling. The model is easily affected by the fluctuation of electromyographic signals when classifying frame by frame, resulting in short-term misjudgment or category jump, which leads to the discontinuity of the action prediction sequence. The Viterbi algorithm constructs the optimal path search strategy based on the state transition probability and the observation probability, which can significantly improve the consistency of the prediction sequence without introducing a complex network structure. Through this decoding process, short-term abnormal points can be corrected while retaining the change trend of the real action, and finally a smoother and more stable result is obtained. Attached Figure Description
[0044] Figure 1 This is a schematic diagram of the overall process of the present invention;
[0045] Figure 2 This is a schematic diagram illustrating the automatic detection of muscle activation through time-frequency energy changes in electromyography signals in an embodiment of the present invention.
[0046] Figure 3 This is a comparison chart of the prediction results before and after smoothing using the Viterbi algorithm in an embodiment of the present invention;
[0047] Figure 4This is a schematic diagram of the confusion matrix for identifying five combined actions in an embodiment of the present invention. Detailed Implementation
[0048] This embodiment provides a method for recognizing the movement intention of upper limb multi-joints based on electromyography signals. Its core is to decouple and model the two main active joints of the upper limb (shoulder joint and elbow joint) into two independent recognition sub-models, and then achieve accurate and stable recognition of complex combined movements through probability fusion and sequence smoothing strategies.
[0049] To achieve the above objectives, such as Figure 1 As shown, the method in this embodiment includes the following steps:
[0050] S1. Acquire electromyographic (sEMG) signals of single joint movements of the upper limb and perform preprocessing. By analyzing the time-frequency energy changes of the signals, automatic detection of the active segment is achieved, thereby assisting manual signal segmentation. The specific steps include the following.
[0051] S1.1 In shoulder joint movement, the main muscle groups involved include the anterior deltoid, middle deltoid, and posterior deltoid. In elbow joint movement, the main muscles involved are the biceps brachii and triceps brachii. As typical antagonistic muscle pairs, they restrain each other during control, which helps to more accurately reflect the joint movement state. Electromyography (EMG) sensors are attached to the muscle groups involved in shoulder and elbow joint movement to obtain EMG signals.
[0052] Electromyographic signals are acquired by an electromyographic sensor and amplified by a signal amplification circuit. To obtain the best signal, the electromyographic sensor is placed in the center of the muscle and parallel to the direction of the muscle fibers, and the raw signal is acquired at a sampling frequency of 2000 Hz.
[0053] S1.2. The acquired electromyographic signals are preprocessed by using a 20-400Hz fourth-order Butterworth bandpass filter to remove high-frequency noise above 400Hz, a 50Hz notch filter to remove power line interference, and a full-wave rectifier and a 1Hz first-order Butterworth low-pass filter to obtain the signal envelope.
[0054] S1.3 Automatic detection of active segments is performed based on the time-frequency energy changes of the signal, ultimately automatically identifying the electromyographic activation segment and generating a label sequence with the same duration as the original signal; such as Figure 2 As shown, the specific steps include the following:
[0055] S1.3.1 Perform continuous wavelet transform (CWT) on the filtered electromyographic signal to obtain the time-frequency signal, and calculate the average value of the time-frequency energy in the active frequency band of 20-150Hz in the frequency dimension to form a time-series energy curve.
[0056] S1.3.2 Acquire an electromyographic signal in a resting state. Perform the previous step to obtain the energy curve in the resting state. The energy curve in the resting state follows an approximate Gaussian distribution, and calculate its mean and standard deviation.
[0057] S1.3.3 Set the activation detection threshold as: Threshold = mean + k·std, where k is an empirical coefficient. In this embodiment, k is 1, which is more sensitive to the signal and suitable for detecting weak activation.
[0058] S1.3.4 Automatically identify electromyographic activation segments and generate a label sequence with the same duration as the original signal.
[0059] S2. Extract the temporal features of the electromyography signal, train the shoulder joint sub-model and elbow joint sub-model respectively, and output the probability distribution of their respective movements.
[0060] For the preprocessed signal, the temporal features of the automatically segmented electromyography (EMG) signal are extracted by sliding window, namely root mean square (RMS), mean absolute value (MAV), wavelength (WL), integral electromyography (IEMG), and slope change (SSC). The EMG signal features of the muscle groups involved in shoulder and elbow joint movements are input into the corresponding shoulder joint sub-model and elbow joint sub-model, respectively.
[0061] The time-domain characteristics of the automatically segmented electromyographic signal are shown in the following formula:
[0062]
[0063] Where, x i Let x represent the signal amplitude at the i-th sampling point, N represent the number of signal points within the sliding window, and x represent the signal amplitude at the ith sampling point. th This represents the noise suppression threshold.
[0064] Based on the basic movement patterns of the shoulder joint commonly used in daily life and rehabilitation applications, the movement patterns of the shoulder include four basic movement patterns: flexion, extension, adduction, and abduction. The movement patterns of the elbow include two basic movement patterns: flexion and extension.
[0065] Both the shoulder joint sub-model and the elbow joint sub-model use XGBoost as the classifier. After training, the output of each sub-model is a probability distribution normalized by softmax, which is used to reflect the model's confidence level for various types of movements.
[0066] S3. The output probability vectors of the shoulder joint sub-model and the elbow joint sub-model are concatenated and then input into the fusion unit to predict the combined action. The fusion unit adopts a multilayer perceptron model. The probability distributions of each single joint action output by the shoulder joint sub-model and the elbow joint sub-model are concatenated into a probability vector and then input into the fusion unit for training. After training, it is used to predict multi-joint combined motion. The fusion unit is trained through supervised learning. The goal is to minimize the cross-entropy loss function between the output probability distribution and the true combined action label.
[0067] The fusion layer consists of an input layer, a first hidden layer, a second hidden layer, and an output layer. The input layer has 6 nodes, which extract the probability output vectors of the two shoulder joint sub-models and the elbow joint sub-model, and concatenate them frame by frame in time to form a joint probability vector P. joint This vector is output by the shoulder model. Elbow model output It is pieced together, that is This vector serves as the input to subsequent connected layers; the first hidden layer includes 32 neurons and a ReLU activation function; the second hidden layer includes 16 neurons and a ReLU activation function; the output layer includes 5 nodes and a softmax activation function. The 5 nodes correspond to 5 types of combined actions. The 5 combined actions are selected from common complex upper limb movements in daily life, including picking up objects from high places, drinking water from a cup, pushing doors, lifting objects, and swinging arms.
[0068] S4. Perform temporal smoothing optimization on the predicted sequence using the Viterbi decoding algorithm, such as... Figure 3 The image shows a comparison of the results of the Viterbi algorithm's smoothing prediction for five combined actions. Step S4 specifically includes the following sub-steps:
[0069] S4.1 After the fusion unit predicts the continuous sliding time windows, it obtains an output matrix over a time series, denoted as:
[0070] O = [o1 o2 … o] T ]
[0071] Among them, o t This represents the combined action result output by the fusion unit in frame t, with a length of T (corresponding to T-type combined actions).
[0072] S4.2 To model the transition probability relationships between combined actions, a finite-state Markov model is constructed, where the set of states represents all categories of combined actions (e.g., S = {s1 ... s2}). T}).
[0073] S4.3 Construct the state transition probability matrix, which is defined as the state s i Transition to state s jThe probability of the action is denoted as A(i,j), which is obtained by statistically analyzing the frequency of action labels in adjacent time windows in the training set.
[0074] S4.4. Using the fusion output sequence O as the input observation sequence, and considering it as the observation result produced in a certain state, apply the Viterbi algorithm to perform optimal path decoding and calculate the state path with the highest probability:
[0075] S * =argmax P(S|O)
[0076] Where P(S|O) represents the probability of the state sequence S occurring given the observation sequence O, and the process of finding the state path S with the highest probability step by step. * .
[0077] The method of this embodiment was verified by collecting five different elbow + shoulder combination movements, such as... Figure 3 As shown, the method in this embodiment was experimentally evaluated on self-collected data. With each sub-model and fusion unit only needing to learn a small number of basic action categories, the accuracy of recognizing combined action intent reached 93.51%. After introducing Viterbi smoothing prediction sequences, the jitter rate of the recognized sequences was significantly reduced, such as... Figure 4 As shown, the final combined action intent recognition accuracy reached 98.58%.
[0078] In summary, the method in this embodiment achieves an upper limb combined action recognition system with high recognition accuracy, good prediction stability, low sample dependence, and strong modular structure. It has good practical value and engineering feasibility, and is suitable for multiple high-demand scenarios such as prosthetic control, rehabilitation training, and human-computer interaction.
[0079] In addition to the embodiments described above, the present invention may have other implementations. All technical solutions formed by equivalent substitution or equivalent transformation fall within the protection scope claimed by the present invention.
Claims
1. A method for recognizing multi-joint movement intentions of the upper limb based on electromyographic signals, characterized in that: Includes the following steps: S1. Acquire electromyographic signals of single joint movements of the upper limb and preprocess them. Analyze the time-frequency energy changes of the signals to automatically detect the active segment. S2. Extract the temporal features of electromyography signals, train the shoulder joint sub-model and elbow joint sub-model respectively, and output the probability distribution of their respective movements. S3. After concatenating the output probability vectors of the shoulder joint sub-model and the elbow joint sub-model, input them into the fusion unit to predict the combined action. S4. Perform temporal smoothing optimization on the predicted sequence using the Viterbi decoding algorithm.
2. The method for recognizing upper limb multi-joint movement intention based on electromyography signals according to claim 1, characterized in that: Step S1 specifically includes the following sub-steps: S1.
1. Attach electromyography (EMG) sensors to the muscle groups involved in shoulder and elbow joint movements to acquire EMG signals. S1.
2. The collected electromyographic signals are preprocessed. A fourth-order Butterworth bandpass filter of 20-400Hz is used to filter out high-frequency noise above 400Hz, and a 50Hz notch filter is used to filter out power line interference. The signal envelope is obtained by passing through a full-wave rectifier and a 1Hz first-order Butterworth low-pass filter. S1.3 Automatically detect the active segment by the time-frequency energy change of the signal, and finally automatically identify the electromyographic activation segment and generate a label sequence with the same duration as the original signal.
3. The method for recognizing upper limb multi-joint movement intention based on electromyography signals according to claim 2, characterized in that: In step S1.1, the electromyographic signal is acquired by an electromyographic signal sensor and amplified by a signal amplification circuit; the electromyographic signal sensor is placed in the center of the muscle and parallel to the direction of the muscle fibers, and the original signal is acquired at a sampling frequency of 2000Hz.
4. The method for recognizing upper limb multi-joint movement intention based on electromyography signals according to claim 2, characterized in that: Step S1.3 specifically includes the following sub-steps: S1.3.1 Perform continuous wavelet transform on the filtered electromyographic signal to obtain the time-frequency signal, and calculate the average value of the time-frequency energy in the active frequency band of 20-150Hz in the frequency dimension to form a time-series energy curve; S1.3.2 Acquire an electromyographic signal in a resting state, perform the previous step to obtain the energy curve in the resting state, and calculate its mean and standard deviation. S1.3.3 Set the activation detection threshold as: Threshold = mean + k·std, where k is an empirical coefficient; S1.3.4 Automatically identify electromyographic activation segments and generate a label sequence with the same duration as the original signal.
5. The method for recognizing upper limb multi-joint movement intention based on electromyography signals according to claim 1, characterized in that: In step S2, the time-domain features of the automatically segmented electromyographic signals are extracted from the preprocessed signals through a sliding window. These features include root mean square value, mean absolute value, wavelength, integral electromyography, and slope change. The electromyographic signal features of the muscle groups involved in the shoulder and elbow joint movements are then input into the corresponding shoulder joint sub-model and elbow joint sub-model, respectively.
6. The method for recognizing upper limb multi-joint movement intention based on electromyography signals according to claim 5, characterized in that: The time-domain characteristics of the automatically segmented electromyographic signal are shown in the following formula: Where, x i Let x represent the signal amplitude at the i-th sampling point, N represent the number of signal points within the sliding window, and x represent the signal amplitude at the ith sampling point. th This indicates the noise suppression threshold.
7. The method for recognizing upper limb multi-joint movement intention based on electromyography signals according to claim 5, characterized in that: In step S2, the shoulder movement includes flexion, extension, adduction, and abduction of the shoulder joint, and the elbow movement includes flexion and extension of the elbow joint. Both the shoulder joint sub-model and the elbow joint sub-model use XGBoost as the classifier. After training, the output of each sub-model is a probability distribution normalized by softmax, which is used to reflect the model's confidence level for various movements.
8. The method for recognizing upper limb multi-joint movement intention based on electromyography signals according to claim 1, characterized in that: In step S3, the fusion unit uses a multilayer perceptron model. It concatenates the probability distributions of each single joint movement output by the shoulder joint sub-model and the elbow joint sub-model into a probability vector, which is then input into the fusion unit for training. After training, the vector is used to predict multi-joint combined movements.
9. The method for recognizing upper limb multi-joint movement intention based on electromyography signals according to claim 8, characterized in that: The fusion unit is trained through supervised learning with the goal of minimizing the cross-entropy loss function between the output probability distribution and the true combined action label. The fusion layer consists of an input layer, a first hidden layer, a second hidden layer, and an output layer. The input layer has 6 nodes, which extract the probability output vectors of the two shoulder joint sub-models and the elbow joint sub-model, and concatenate them frame by frame in time to form a joint probability vector P. joint This vector is output by the shoulder model. Elbow model output It is pieced together, that is The first hidden layer consists of 32 neurons and a ReLU activation function; the second hidden layer consists of 16 neurons and a ReLU activation function; the output layer consists of 5 nodes and a softmax activation function. The 5 nodes correspond to 5 types of combined actions, including picking up objects from a height, drinking water from a cup, pushing a door, lifting objects, and swinging an arm.
10. The method for recognizing upper limb multi-joint movement intention based on electromyography signals according to claim 1, characterized in that: Step S4 specifically includes the following sub-steps: S4.1 After the fusion unit predicts the continuous sliding time windows, it obtains an output matrix over a time series, denoted as: O=[o1 o2 … o T ] Among them, o t This represents the combined action result output by the fusion unit in frame t; S4.2 Construct a finite-state Markov model to model the transition probability relationships between combined actions, where the state set represents all combined action categories (e.g., S = {s1 ... s2}). T }); S4.3 Construct the state transition probability matrix, which is defined as the state s i Transition to state s j The probability of the action is denoted as A(i,j), which is obtained by statistically analyzing the frequency of action labels in adjacent time windows in the training set. S4.
4. Using the fusion output sequence O as the input observation sequence, and considering it as the observation result produced in a certain state, apply the Viterbi algorithm to perform optimal path decoding and calculate the state path with the highest probability: S * =argmax P(S|O) Where P(S|O) represents the probability of the state sequence S occurring given the observation sequence O, and the process of finding the state path S with the highest probability step by step. * .