Motion intention decoding method and system

Abstract

The invention discloses a motion intention decoding method and system. The method comprises the following steps: acquiring first signal data; based on the first signal data and a prediction model, a corresponding prediction result is generated, and the prediction result is used for indicating the motion intention corresponding to the next action of the user; based on the first signal data and an intention recognition model, a corresponding decoding result is generated and output, and the decoding result is used for indicating a motion intention corresponding to the current action of the user; calculating a deviation degree between the decoding result and the prediction result; and when the deviation degree exceeds a preset deviation threshold value, performing reverse updating on the intention recognition model based on the deviation degree. While the current action is decoded, prospective prediction is performed on the next action, and the intention recognition model is automatically optimized based on the deviation condition of the prediction result and the decoding result, so that the intention recognition model is suitable for a complex collaborative task or a multi-stage task.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a motion intention decoding method and system. Background Art

[0002] Existing methods mainly rely on real-time collection of physiological signals for decoding, which makes it difficult to meet the needs of complex collaborative tasks or multi-stage tasks.

[0003] Physiological signals include: Electroencephalogram (EEG); Electromyography (EMG); Functional Near-Infrared Spectroscopy (fNIRS); Motion and Posture Signals (MPS). Summary of the Invention

[0004] In view of the shortcomings of the motion intention decoding method disclosed in the prior art that is not suitable for complex collaborative tasks or multi-stage tasks, the present invention proposes a motion intention decoding method and system.

[0005] In order to solve the above technical problems, the present invention is solved by the following technical solutions: A motion intention decoding method comprises the following steps: Acquire first signal data, where the first signal data includes one or more physiological signals corresponding to a current exercise period of the user; generating a corresponding prediction result based on the first signal data and the prediction model, wherein the prediction result is used to indicate the movement intention corresponding to the user's next action; Based on the first signal data and the intention recognition model, a corresponding decoding result is generated and output, wherein the decoding result is used to indicate the movement intention corresponding to the user's current action; Calculating a degree of deviation between the decoding result and the prediction result; When the deviation exceeds a preset deviation threshold, the intention recognition model is reversely updated based on the deviation.

[0006] This embodiment is applicable to scenarios based on single physiological signal decoding or multimodal signal fusion decoding; This embodiment makes a forward-looking prediction of the user's next action while decoding, and when the deviation between the prediction result and the decoding result is large, the deviation is used as a feedback signal to optimize the intention recognition model. Even if the difference between the previous and subsequent actions is large, the accuracy of decoding can be guaranteed.

[0007] As an implementable approach: The first signal data includes a first physiological signal and a plurality of second physiological signals, wherein the first physiological signal is an electroencephalogram signal; The intent recognition model includes a unimodal decoding model and a multimodal decoding model. When the deviation between the corresponding decoding result and the prediction result exceeds a preset deviation threshold, the unimodal decoding model and the multimodal decoding model are reversely updated based on the deviation; The specific steps of generating a corresponding decoding result based on the first signal data and the intent recognition model include: comparing the signal quality score corresponding to the first physiological signal with a preset first score threshold; When the signal quality score corresponding to the first physiological signal reaches the first score threshold, decoding the first physiological signal based on the single-modality decoding model to obtain a corresponding decoding result; When the signal quality score corresponding to the first physiological signal does not reach the first scoring threshold, the first physiological signal is used as the target decoding signal, and the second physiological signal with a signal quality score higher than the second scoring threshold is used as the target decoding signal. The multimodal decoding model performs multimodal fusion decoding based on each target decoding signal to obtain a corresponding decoding result.

[0008] This embodiment uses the EEG signal as the main decoding signal. When the signal quality of the EEG signal is good, single-modal decoding is performed based on the EEG signal. When the signal quality of the EEG signal is poor, other physiological signals with better signal quality are used as supplements to perform multi-modal decoding. This embodiment can intelligently select an appropriate signal decoding method, which can not only improve the accuracy and robustness of decoding, but also effectively improve the real-time and adaptability of the system.

[0009] As an implementable approach: The second physiological signal is an electromyographic signal, a functional near-infrared signal and a posture signal.

[0010] As an implementable approach: The prediction model is a digital twin prediction model; The input of the digital twin prediction model is EEG signals, EMG signals and posture signals, and the output is the corresponding prediction results.

[0011] As an implementable approach: The specific steps of performing multimodal fusion decoding based on each target decoding signal by the multimodal decoding model are as follows: Extracting decoding features corresponding to each target decoding signal; Perform feature weighting processing based on the signal quality score and decoding features corresponding to each target decoding signal to obtain the corresponding fusion feature; The fusion feature is input into the multimodal decoding model, and the multimodal decoding model outputs a corresponding decoding result.

[0012] As an implementation method, feature weighting processing is performed based on the signal quality score and decoding features corresponding to each target decoded signal to obtain the corresponding fusion features in the following specific steps: Get the decoding requirements corresponding to the current decoding task; When the decoding requirement is real-time, a first feature fusion model performs weighted fusion based on the signal quality score corresponding to the target decoded signal and the fusion feature to obtain a corresponding fusion feature, wherein the first feature fusion model is a lightweight feature fusion model; When the decoding requirement is accuracy, the second feature fusion model performs weighted fusion based on the signal quality score and fusion features corresponding to the target decoding signal to obtain the corresponding fusion features. The second feature fusion model is a deep learning model.

[0013] The decoding requirement is a configuration item, which can be pre-configured or dynamically adjusted during the intent recognition process. In this embodiment, a suitable feature fusion model is automatically selected based on the configured decoding requirement to further optimize the decoding effect.

[0014] As an implementable method, a method for calculating the signal quality score corresponding to the first physiological signal is: A corresponding signal quality score is calculated based on the signal-to-noise ratio, spectral entropy, and power ratio of the first physiological signal.

[0015] As an implementable embodiment, a method for calculating the signal quality score corresponding to each second physiological signal includes: Based on the root mean square value, median frequency of the myoelectric spectrum and waveform complexity of the electromyographic signal, the corresponding signal quality score is calculated; Based on the signal amplitude change, optical signal intensity and spectral characteristics corresponding to the functional near-infrared signal, the corresponding signal quality score is calculated; Based on the acceleration mean deviation, angular velocity stability, and signal sampling rate stability corresponding to the attitude signal, the corresponding signal quality score is calculated.

[0016] The present invention also proposes a motion intention decoding system, comprising: an acquisition module, configured to acquire first signal data, wherein the first signal data includes one or more physiological signals corresponding to a current exercise period of the user; A prediction system, configured to generate a corresponding prediction result based on the first signal data and the prediction model, wherein the prediction result is used to indicate the movement intention corresponding to the user's next action; A decoding system, configured to generate and output a corresponding decoding result based on the first signal data and the intention recognition model, wherein the decoding result is used to indicate the movement intention corresponding to the user's current action; A feedback system is used to calculate the deviation between the decoding result and the prediction result, and is also used to reversely update the intention recognition model based on the deviation when the deviation exceeds a preset deviation threshold.

[0017] The present invention further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, any of the above methods is implemented.

[0018] The present invention has significant technical effects due to the adoption of the above technical solutions: While performing real-time decoding based on the first signal data, the present invention also makes a forward-looking prediction of the user's next action. When the deviation between the decoding result of the real-time action and the predicted result of the next action is large, the deviation is used as a feedback signal to optimize the intention recognition model, which can improve the decoding accuracy when the next action is decoded in real time. This design can effectively ensure the accuracy of motion intention decoding in scenarios of complex collaborative tasks or multi-stage tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 1 is a schematic diagram of the workflow of a motion intention decoding method shown in Embodiment 1 and Embodiment 2 of the present invention; Figure 2 3 is a schematic diagram of the workflow of a motion intention decoding method shown in Example 3 of the present invention; Figure 3 This is a schematic diagram of the workflow of a motion intention decoding method shown in Example 4 of the present invention. DETAILED DESCRIPTION

[0021] Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numerals in different drawings represent the same or similar elements.

[0022] The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. Instead, they are only examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0023] It should be noted that in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the method may include more or fewer steps than those described in this specification. In addition, a single step described in this specification may be broken down into multiple steps for description in other embodiments, and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0024] Example 1: A method for decoding movement intention based on a single physiological signal. Figure 1 , including the following steps: A100, obtaining first signal data; The first signal data is a physiological signal corresponding to the user's current exercise period; Those skilled in the art may select a suitable physiological signal as the first signal data according to actual needs. In this embodiment, the first signal data is an electroencephalogram signal.

[0025] A200, generating a corresponding prediction result based on the first signal data and the prediction model; The prediction result is used to indicate the movement intention corresponding to the user's next action; In this embodiment, the input of the prediction model is the EEG signal, and the output is the prediction result corresponding to the next action.

[0026] In determining the learning goal of the first signal data and predicting the motion intention corresponding to the next action, those skilled in the art can construct the corresponding prediction model based on the existing technology, and this specification does not limit it in detail.

[0027] A300, based on the first signal data and the intention recognition model, generates and outputs a corresponding decoding result; The decoding result is used to indicate the movement intention corresponding to the user's current action; In this embodiment, the intention recognition model is a unimodal decoding model. The input of the unimodal decoding model is the corresponding feature data, and the output is the decoding result corresponding to the current action. The feature data is the decoding feature corresponding to the first signal data, namely the EEG signal.

[0028] Specifically: Extract features from the EEG signal to obtain corresponding decoding features. In this embodiment, the decoding features include frequency domain features (such as alpha wave and beta wave energy), spectral entropy (SE), and time domain features (such as P300 waveform) corresponding to the EEG signal. The decoding features are input into a unimodal decoding model, and the unimodal decoding model outputs a corresponding decoding result.

[0029] As above, in determining the first signal data and the learning goal of identifying the motion intention of the current action, those skilled in the art can obtain the corresponding unimodal decoding model based on existing technical training, which is not limited in detail in this specification.

[0030] A400: Calculate the degree of deviation between the decoding result and the prediction result.

[0031] In this embodiment, the decoding result is a vector and the prediction result is a scalar. When calculating the degree of deviation, the calculation can be performed directly based on the magnitudes of the decoding result and the prediction result. For example, those skilled in the art can determine the degree of deviation by using the Mean Absolute Error (MAE). In this embodiment, based on a step size of step and a sliding window of n, n pairs of decoding results and prediction results are obtained to calculate the deviation degree. Those skilled in the art can set the values ​​of step and n according to actual needs. Specifically, the average value of the absolute errors of n consecutive groups of decoding results and prediction results (including the current decoding result and prediction result) is calculated as the deviation degree. The calculation formula is as follows.

[0032]

[0033] in: Prediction To predict the results, Decoding The value of the decoding result.

[0034] A500. When the deviation exceeds a preset deviation threshold, the intention recognition model is reversely updated based on the deviation.

[0035] When the deviation exceeds the preset deviation threshold, it means that the difference between the current action and the next action is large. If the current intention recognition model is continued to be used in the next step, the decoding effect is not good. In this embodiment, the deviation is used as a feedback signal, and the model parameters of the intention recognition model are optimized based on the deviation. When identifying the next action, the motion intention decoding will be performed based on the updated intention recognition model, so that the intention recognition model can perform more accurate intention decoding based on the user's individual characteristics and dynamic changes.

[0036] Example 2: A motion intention decoding method based on multimodal physiological signal fusion decoding; In this embodiment: The first signal data includes several physiological signals; The input of the prediction model is part or all of the physiological signals in the first signal data, and the output is the prediction result corresponding to the next action; The intention recognition model is a multimodal decoding model, the input of which is the corresponding fusion feature, and the output is the decoding result corresponding to the current action.

[0037] In this embodiment, the features corresponding to each physiological signal are fused based on the feature fusion model to obtain the fused feature. That is, the input of the feature fusion model is the features corresponding to each physiological signal in the first signal data, and the output is the corresponding fused feature.

[0038] Reference Figure 1 , as an embodiment; B100, obtaining first signal data; In this embodiment, the first signal data includes an electroencephalogram signal, an electromyography signal, a functional near-infrared signal, and a posture signal; Those skilled in the art may select a variety of physiological signals as the first signal data according to actual needs.

[0039] B200, generating a corresponding prediction result based on the first signal data and the prediction model; B210. Using part or all of the physiological signals in the first signal data as second signal data; In this embodiment, the EEG signal, EMG signal and posture signal are used as the second signal data; Those skilled in the art can determine the number and types of physiological signals in the second signal data based on the first signal data according to actual needs.

[0040] B220, pre-build prediction models; Those skilled in the art may, according to actual needs, construct a corresponding prediction model on their own based on the first signal data and the learning goal of predicting the motion intention corresponding to the next action.

[0041] In this embodiment, the prediction model is a pre-built digital twin prediction model. The method for constructing the digital twin prediction model includes: Obtain sample training data, which can collect EEG signals, EMG signals and posture signals of users during exercise in advance to obtain corresponding sample training data; Use reinforcement learning (RL) algorithms, such as model predictive control (MPC) algorithms; The twin model is trained using the sample training data to obtain a digital twin prediction model that can simulate the individual intention pattern of the target user. In real-time tasks, the digital twin prediction model is used to predict the target user's next possible intention or action.

[0042] B230. Input the second signal data into a pre-built digital twin prediction model, and the digital twin prediction model outputs a corresponding prediction result.

[0043] B300, based on the first signal data and the intention recognition model, generating and outputting a corresponding decoding result; Specifically: B310, extracting features of each physiological signal respectively; In this embodiment: Extract frequency domain features (such as α wave and β wave energy), spectral entropy (SE), and time domain features (such as P300 waveform) corresponding to the EEG signal; Extract features such as the root mean square (RMS) and spectral power density corresponding to the electromyographic signal; Extract features such as changes in oxygenated hemoglobin (HbO) and deoxygenated hemoglobin (HbR) corresponding to functional near-infrared signals; The features corresponding to the posture signal, such as Mean Deviation of Acceleration (MDA), Rate of Acceleration Change (RAC), Cumulative Rotation Angle (CRA), Tilt Angle (TA) and Step Frequency (SF), are extracted.

[0044] B320, feature fusion; Based on the feature fusion model, the features of each physiological signal are fused to obtain the corresponding fusion features.

[0045] Those skilled in the art can select an existing disclosed feature fusion model according to actual needs.

[0046] B330, motion intention decoding: The obtained fusion features are input into the multimodal decoding model, and the multimodal decoding model outputs the corresponding decoding results.

[0047] When the input of the model is determined to be fusion features and the learning goal is to decode and obtain the motion intention of the current action, those skilled in the art can obtain the corresponding multimodal decoding model based on existing technical training, and this specification does not limit it in detail.

[0048] B400. Calculate the degree of deviation between the decoding result and the prediction result.

[0049] B500: When the deviation exceeds a preset deviation threshold, reversely update the multimodal decoding model based on the deviation.

[0050] Example 3: A motion intention decoding method, referring to Figure 2 , the step of “generating a corresponding decoding result based on the first signal data and the intent recognition model” in Example 2 is further optimized, and the rest is the same as Example 2; In this embodiment, the EEG signal is used as the first bio-signal, and the myoelectric signal, functional near-infrared signal and posture signal are used as the second bio-signal; In this embodiment, the intent recognition model includes a unimodal decoding model and a multimodal decoding model; The optimized decoding steps are as follows: C100, comparing the signal quality score corresponding to the first physiological signal with a preset first score threshold; Those skilled in the art can set the quality scoring rules corresponding to the first physiological signal (i.e., EEG signal) according to actual needs; In this embodiment, the signal quality score corresponding to the EEG signal is calculated based on the signal-to-noise ratio, spectral entropy and power ratio corresponding to the EEG signal. The calculation formula is: Q EEG =( eeg W1 × SNR norm + eeg W2 × SE norm + eeg W3 × ABR norm ) × 100; in: Q EEG Indicates the signal quality score corresponding to the EEG signal; eeg W1 、 eeg W2 、 eeg W3 is the feature weight corresponding to each decoding feature of the EEG signal, which can be set by those skilled in the art according to actual needs. eeg W1 + eeg W2 + eeg W3 =1, in this embodiment eeg W1 =0.4, eeg W2 =0.3 、eeg W3 =0.3; SNR norm 、 SE norm 、 ABR norm is the eigenvalue of the corresponding feature after normalization, ranging from [0 1]; SNR norm represents the corresponding signal-to-noise ratio eigenvalue; SE norm represents the corresponding spectral entropy eigenvalue; ABR norm represents the corresponding power ratio eigenvalue.

[0051] The following is a detailed introduction to how to obtain features such as the signal-to-noise ratio, spectral entropy, and power ratio of EEG signals: Signal-to-Noise Ratio (SNR): It indicates the energy ratio of the effective component of the corresponding signal to the noise, and can reflect the clarity of the signal quality. The calculation formula is: .

[0052] Spectral Entropy (SE): It is used to indicate the distribution complexity of the corresponding spectrum. When its value is low, it means that the signal is more concentrated, and when it is high, it means that there may be more noise interference. The calculation formula is: ; in, p(f) is the normalized power spectrum density, which is used to represent the power distribution of the signal at different frequencies.

[0053] Power ratio (Alpha / Beta Ratio, ABR): in EEGα Waves (8-13 Hz) and β wave (13-30 Hz) energy ratio; A high power value indicates a state of relaxation, while a low power value reflects artifact interference or a task state. The calculation formula is as follows: ; in: α power express α wave energy; β power express β Wave energy.

[0054] C200, when the signal quality score corresponding to the first physiological signal Q EEG Reaching the first scoring threshold Q Thr1 When the first physiological signal is decoded in a single mode, a corresponding decoding result is obtained; That is, when Q EEG ≥ Q Thr1 When , it indicates that the EEG signal quality is high and can be used alone for intent decoding.

[0055] Those skilled in the art can set the first scoring threshold according to actual needs. Q Thr1 , this embodiment does not limit it in detail.

[0056] In this embodiment, the first physiological signal, namely the EEG signal, is decoded by a single-modality decoding model to obtain a corresponding decoding result, specifically: C210, extracting a decoding feature corresponding to the first physiological signal; In this embodiment, the decoding features corresponding to the first physiological signal include frequency domain features (such as alpha wave and beta wave energy), spectral entropy (SE), and time domain features (such as P300 waveform); C220: Input the obtained decoding features into a unimodal decoding model, and the unimodal decoding model outputs a corresponding decoding result.

[0057] C300, when the signal quality score corresponding to the first physiological signal Q EEG Does not meet the first scoring threshold Q Thr1When the first physiological signal is used as the target decoding signal, and the second physiological signal whose signal quality score is higher than the second score threshold is used as the target decoding signal, multimodal fusion decoding is performed on each target decoding signal; That is, when Q EEG < Q Thr1 When , it indicates that the signal quality of the EEG signal is poor and needs to be combined with other signals for multimodal fusion decoding.

[0058] The specific steps are: C310, taking the second physiological signal whose signal quality score reaches a preset second score threshold as the target decoding signal; Those skilled in the art can set the quality scoring rules corresponding to the second physiological signal (i.e., electromyographic signal, functional near-infrared signal, and posture signal) according to actual needs; For electromyography (EMG): In this embodiment, the signal quality score corresponding to the electromyographic signal EMG is calculated based on the root mean square value, electromyographic spectrum median frequency and waveform complexity corresponding to the electromyographic signal EMG. The calculation formula is: Q EMG =( emg W1 × RMS norm + emg W2 × MPF norm + emg W3 × ZCR norm ) × 100; in: Q EMG Indicates the signal quality score corresponding to the electromyographic signal EMG; emg W1 、 emg W2 、 emg W3 is the feature weight corresponding to each decoding feature of the electromyographic signal EMG, which can be set by those skilled in the art according to actual needs. emg W1 + emg W2 + emg W3 =1, in this embodiment emg W1 =0.4, emg W2=0.4, emg W3 =0.2; RMS norm 、 MPF norm 、 ZCR norm is the eigenvalue of the corresponding feature after normalization, ranging from [0 1]; RMS norm represents the corresponding RMS eigenvalue; MPF norm Indicates the corresponding median frequency eigenvalue of the electromyographic spectrum; ZCR norm Indicates the corresponding waveform complexity eigenvalue.

[0059] The following is a detailed introduction to how to obtain features such as the root mean square value of the electromyographic signal, the median frequency of the electromyographic spectrum, and the complexity of the waveform: Root Mean Square (RMS): reflects the energy characteristics of the electromyographic signal. When the noise ratio is high, the RMS value is low. The calculation formula is as follows:

[0060] in, N represents the total number of sampling points, x i For the i The signal sampling value corresponding to the sampling point.

[0061] Median Frequency (MDF): This represents the frequency distribution characteristics of the EMG power spectrum. A lower median frequency indicates artifacts or fatigue. In this embodiment, the power spectrum density corresponding to the EMG signal is calculated, and the frequency at which the power integration of the frequency range is half of the total power is used as the median frequency (MDF) of the EMG spectrum.

[0062] Waveform complexity (Zero Crossing Rate, ZCR): used to reflect the change pattern of the electromyographic signal. In this embodiment, the waveform complexity is determined based on the signal zero crossing rate (ZCR) per unit time.

[0063] Targeting functional near-infrared signals (fNIRS): In this embodiment, the signal quality score corresponding to the functional near-infrared signal is calculated based on the signal amplitude change, optical signal intensity and spectral characteristics corresponding to the functional near-infrared signal. The calculation formula is: QfNIRS =( fnirs W1 × AV norm + fnirs W2 × OSI norm + fnirs W3 × FF norm ) × 100; in: Q fNIRS It represents the signal quality score corresponding to the functional near-infrared signal fNIRS; fnirs W1 、 fnirs W2 、 fnirs W3 is the feature weight corresponding to each decoding feature of functional near-infrared signal fNIRS, which can be set by those skilled in the art according to actual needs. fnirs W1 + fnirs W2 + fnirs W3 =1, in this embodiment fnirs W1 =0.4, fnirs W2 =0.4, fnirs W3 =0.2; AV norm 、 OSI norm 、 FF norm is the eigenvalue of the corresponding feature after normalization, ranging from [0 1]; AV norm Indicates the corresponding signal amplitude change characteristic value; OSI norm Indicates the corresponding optical signal intensity characteristic value; FF norm Indicates the corresponding spectrum eigenvalue.

[0064] The following is a detailed introduction to how to obtain features such as signal amplitude changes, optical signal intensity, and spectral power corresponding to functional near-infrared signals: Signal Amplitude Variability (AV): used to indicate the amplitude of the change in oxygenated hemoglobin (HbO) and deoxygenated hemoglobin (HbR) in the corresponding signal over time. In this embodiment, it is obtained by calculating the difference between the peak and valley values ​​of the corresponding signal.

[0065] Optical Signal Intensity (OSI): This embodiment uses a detector to collect the corresponding light intensity and calculates the average value, which is used as the optical signal intensity. A low optical signal intensity may indicate a high signal-to-noise ratio. The calculation formula is as follows:

[0066] in: T represents the total number of corresponding sampling points I(t) is the light intensity corresponding to the t-th sampling point.

[0067] Frequency Features (FF): These are used to analyze the low-frequency components (e.g., 0.01–0.2 Hz) of the HbO and HbR signals to determine whether they conform to typical neurovascular coupling patterns. In this embodiment, the spectral power of the corresponding signals is extracted based on the fast Fourier transform (FFT) to obtain the corresponding spectral features.

[0068] For attitude signal MPS: In this embodiment, the signal quality score corresponding to the posture signal MPS is calculated based on the acceleration mean deviation, angular velocity stability, and signal sampling rate stability corresponding to the posture signal MPS. The calculation formula is: Q MPS =( mps W1 × MDA norm + mps W2 × AVS norm + mps W3 × SRS norm ) × 100; in: Q MPS Indicates the signal quality score corresponding to the posture signal MPS; mps W1 、 mps W2 、 mps W3is the feature weight corresponding to each decoded feature of the posture signal MPS, which can be set by those skilled in the art according to actual needs. mps W1 + mps W2 + mps W3 =1, in this embodiment mps W1 =0.4, mps W2 =0.4, mps W3 =0.2; MDA norm 、 AVS norm 、 SRS norm is the eigenvalue of the corresponding feature after normalization, ranging from [0 1]; MDA norm represents the corresponding acceleration mean deviation eigenvalue; AVS norm represents the corresponding angular velocity stability eigenvalue; SRS norm Indicates the corresponding signal sampling rate stability characteristic value.

[0069] The following is a detailed introduction to how to obtain characteristics such as acceleration mean deviation, angular velocity stability, and signal sampling rate stability corresponding to the attitude signal: Mean Deviation of Acceleration (MDA): This is the average deviation of the three-axis acceleration from its baseline value. The calculation formula is as follows:

[0070] in: M represents the total number of corresponding sampling points; a i Indicates the i The three-axis acceleration corresponding to each sampling point; a 0 Indicates the preset three-axis acceleration reference value; Angular Velocity Stability (AVS): That is, the rate of change of the angular velocity of the gyroscope, calculated as follows:

[0071] in: L represents the total number of corresponding sampling points; △w i Indicates the i The angular velocity corresponding to each sampling point; △t Indicates unit time; Sampling Rate Stability (SRS): used to indicate whether the sampling interval of the corresponding signal is uniform. In this embodiment, it is calculated by detecting the interval change between consecutive time points.

[0072] As another possible implementation method, when the signal quality scores corresponding to all second physiological signals do not reach the preset second score threshold, the decoding mode is switched to single physiological signal decoding, that is, the corresponding EEG signal is decoded based on a single-modal decoding model.

[0073] As an implementable embodiment, when the signal quality scores corresponding to all second physiological signals do not reach the preset second score threshold, poor signal quality is fed back to the user, and the user is shown the correct wearing method of the relevant device.

[0074] Note: Those skilled in the art may set the node for calculating the signal quality score of each second physiological signal according to actual needs. The quality score of each second physiological signal may be performed after determining that the signal quality score corresponding to the first physiological signal does not reach a preset first score threshold. Alternatively, the scoring of the second physiological signal may be completed immediately after the second physiological signal is obtained. This embodiment does not limit this in detail. Those skilled in the art can set the second scoring threshold according to actual needs. Q Thr2 , this embodiment does not limit it in detail.

[0075] C320. Based on the obtained target decoding signal, perform intent decoding through the multimodal decoding model to obtain a corresponding decoding result.

[0076] The specific steps include: C321, extracting decoding features corresponding to each target decoding signal; C322. Perform feature weighting processing based on the signal quality score and decoding features corresponding to each target decoded signal to obtain corresponding fusion features; In this embodiment, the feature fusion model performs weighted processing of multimodal signal features based on the signal quality scores and fusion features corresponding to each target decoded signal, and outputs the corresponding feature matrix as the fusion feature. The corresponding weights in the feature fusion process are dynamically adjusted according to the signal quality scores. Those skilled in the art may select a deep learning model, a random forest model, or a mean-weighted model as a feature fusion model according to actual needs, and this embodiment does not limit it in detail.

[0077] C323. Input the fusion features into a multimodal decoding model, and the multimodal decoding model outputs a corresponding decoding result.

[0078] In this embodiment: The frequency domain features (such as α wave and β wave energy), spectral entropy (SE) and time domain features (such as P300 waveform) corresponding to the EEG signal are used as its decoding features; The root mean square (RMS) and spectral power density of the electromyographic signal are used as decoding features. The change characteristics of oxygenated hemoglobin (HbO) and deoxygenated hemoglobin (HbR) corresponding to the functional near-infrared signal are used as its decoding features; The features corresponding to the posture signal, such as the mean deviation of acceleration (MDA), rate of acceleration change (RAC), cumulative rotation angle (CRA), tilt angle (TA) and step frequency (SF), are used as decoding features.

[0079] In this embodiment, when the deviation between the corresponding decoding result and the prediction result exceeds a preset deviation threshold, the single-modal decoding model and the multi-modal decoding model are reversely updated based on the deviation to improve the decoding accuracy during the next motion intention decoding.

[0080] Current motion intent decoding technology relies primarily on the following two methods: Based on single physiological signal decoding: This method directly decodes movement intention through signals such as electroencephalogram (EEG), electromyography (EMG), functional near-infrared spectroscopy (fNIRS) or motion and posture signals (MPS). This method is simple to implement, but when the signal quality is low, the noise is high, or the individual differences are large, the decoding performance often decreases significantly, affecting the accuracy of intent recognition.

[0081] Based on multimodal signal fusion decoding: That is, multimodal signals such as EEG, EMG, and MPS are combined to comprehensively decode movement intention; This method improves the decoding accuracy and robustness to a certain extent, but its real-time performance and practicality are limited due to the complexity of the data fusion algorithm and its high requirements on computing resources.

[0082] In summary, the single physiological signal decoding method is highly dependent on signal quality and human functional status. When the signal noise is high or the motor function is severely impaired, its decoding performance decreases significantly and its scope of application is relatively limited. Although multimodal signal fusion decoding improves robustness, it lacks real-time and convenience. Data processing delays in complex scenarios may affect the experience.

[0083] To address the above-mentioned shortcomings, this embodiment intelligently selects an appropriate signal decoding method based on the characteristics (decoding features) and quality (signal quality score) of various physiological signals, which not only improves the accuracy and robustness of decoding, but also effectively improves the real-time performance and adaptability of the system.

[0084] Example 4: A motion intention decoding method, referring to Figure 3 , the step of "performing feature fusion based on the signal quality score and decoding features corresponding to each target decoded signal to obtain corresponding fusion features" in Example 3 is further optimized, and the rest is the same as Example 3; In this embodiment, the feature fusion model includes a first feature fusion model and a second feature fusion model; The inputs of the first feature fusion model and the second feature fusion model are both the signal quality scores and fusion features corresponding to the target decoded signals, and the outputs are both the corresponding fusion features; The first feature fusion model is an existing lightweight feature fusion model such as a weighted average model or a random forest model; The second feature fusion model is an existing publicly available deep learning model such as a Transformer network based on an attention mechanism.

[0085] The optimized fusion steps are as follows: D100, obtain the decoding requirements corresponding to the current decoding task; Decoding requirements include real-time performance and accuracy; users can set and adjust decoding requirements based on actual conditions.

[0086] D200. When the decoding requirement is real-time, the first feature fusion model performs weighted fusion based on the signal quality score corresponding to the target decoding signal and the fusion feature to obtain the corresponding fusion feature.

[0087] D300. When the decoding requirement is accuracy, the second feature fusion model performs weighted fusion based on the signal quality score corresponding to the target decoding signal and the fusion feature to obtain the corresponding fusion feature.

[0088] This embodiment, through the design of decoding requirements, can dynamically switch the fusion strategy according to the complexity of the task and the real-time requirements during the decoding process; For example, for high-precision tasks (such as decoding fine hand movements), a deep learning fusion model is used; while for tasks with high real-time requirements (such as gait recognition), lightweight models such as weighted averaging or random forest are used.

[0089] In summary, the motion intention recognition method provided in this embodiment comprehensively selects decoding algorithms and fusion strategies through signal quality, feature distribution and real-time requirements; among them, the signal quality index reflects the noise level and reliability of the signal, the feature distribution index reflects the expression pattern of the motion intention in the signal, and the real-time requirement ensures that the decoding system can meet the dynamic adaptation of complex task scenarios; based on the comprehensive evaluation and algorithm selection of these key indicators, it can achieve accurate recognition of user intentions while ensuring the robustness of decoding and the efficiency of the system, further ensuring the stable performance and excellent effect of the motion intention recognition technology in a variety of application scenarios.

[0090] In this embodiment, the corresponding first feature fusion model, second feature fusion model, single-modal decoding model and multi-modal decoding model are pre-trained. In actual applications, the parameters of the single-modal decoding model and the multi-modal decoding model are optimized based on the deviation between the decoding result and the prediction result corresponding to each decoding task.

[0091] In order to improve the universality and robustness of the motion intention recognition method, a signal database and decoding model library are pre-built in this embodiment: Signal database: By collecting EEG, EMG, fNIRS, and MPS signals from a large number of users of different ages, genders, health conditions, and task scenarios, a standardized multimodal signal database is constructed. The signal database provides signal quality assessment indicators and features, providing a basis for signal quality grading and signal weighting strategies in real-time tasks.

[0092] Decoding model library: Based on the data samples provided by the signal database, deep learning, random forest and other algorithms are used to train single-modal decoding models, as well as a model combination of the first feature fusion model, the second feature fusion model and the multimodal decoding model, and to cover task requirements from motion recognition to cognitive state analysis.

[0093] By grading the signal quality scores and designing the decoding requirements, this method has an adaptive selection mechanism, which can dynamically select the optimal model combination according to the real-time signal quality and decoding requirements, ensuring decoding efficiency and robustness in different scenarios.

[0094] The following uses a specific case to illustrate the application of the motion intention recognition method proposed in this embodiment; During rehabilitation training, the user wears a brain-computer interface device, and the method identifies the user's movement intention based on the collected physiological signals, and controls the corresponding rehabilitation robot to complete the limb training task based on the movement intention; Users and rehabilitation doctors can adjust the intensity and mode of training tasks by observing the synchronization between decoding results and actual movements; this real-time feedback mechanism effectively improves the accuracy and efficiency of rehabilitation training, while enhancing patients' understanding and participation in the training process.

[0095] The system also displays the signal quality score for each signal in real time. In remote guidance scenarios, rehabilitation physicians can provide real-time feedback to users based on the signal quality score, real-time decoding results, and digital twin prediction results, allowing them to adjust training plans or related parameters (such as the parameters of the digital twin prediction model, the feature weights corresponding to each physiological signal during quality assessment, and decoding requirements). For example, if an abnormality is detected in a user's modality signal (such as fNIRS), the physician can promptly advise the user to optimize the wearing method or adjust the training environment.

[0096] Embodiment 5: A motion intention recognition system, comprising: an acquisition module, configured to acquire first signal data, wherein the first signal data includes one or more physiological signals corresponding to a current exercise period of the user; A prediction system, configured to generate a corresponding prediction result based on the first signal data and the prediction model, wherein the prediction result is used to indicate the movement intention corresponding to the user's next action; A decoding system, configured to generate and output a corresponding decoding result based on the first signal data and the intention recognition model, wherein the decoding result is used to indicate the movement intention corresponding to the user's current action; The intent recognition model is a unimodal decoding model or a multimodal decoding model; A feedback system is used to calculate the deviation between the decoding result and the prediction result, and is also used to reversely update the intention recognition model based on the deviation when the deviation exceeds a preset deviation threshold.

[0097] Example 6: A motion intention recognition system, which further optimizes the decoding system based on Example 5, and is otherwise identical to Example 5; In this embodiment, the first signal data includes a first physiological signal and several second physiological signals, wherein the first physiological signal is an electroencephalogram signal; and the second physiological signal is an electromyography signal, a functional near-infrared signal, and a posture signal.

[0098] The decoding system includes: A quality assessment model is used to calculate the signal quality score corresponding to each physiological signal; a judgment module, configured to compare the signal quality score of the first physiological signal with a preset first scoring threshold, and further configured to compare the signal quality score of each second physiological signal with a preset second scoring threshold; a feature extraction module, configured to extract decoding features of a target decoding signal, and when a signal quality score of a first physiological signal reaches a preset first scoring threshold, use the first physiological signal as the target decoding signal; otherwise, use both the first physiological signal and a second physiological signal whose signal quality score reaches a preset second scoring threshold as the target decoding signal; a single-modality decoding module, which, when there is only one target decoding signal, inputs the decoding features corresponding to the first physiological signal into a single-modality decoding model, and the single-modality decoding model outputs a corresponding decoding result; A fusion module, when there are multiple target decoding signals, performs feature fusion based on the signal quality scores and decoding features corresponding to each target decoding signal to obtain a fusion feature; The multimodal decoding module inputs the fusion features into the single-modal decoding model, and the multimodal decoding model outputs the corresponding decoding results.

[0099] Example 7: A motion intention recognition system. Based on Example 6, the fusion module is further optimized, and the rest is the same as Example 6. The fusion module includes: A first feature fusion model is used to perform weighted fusion based on the signal quality score corresponding to the target decoded signal and the fusion feature to obtain a corresponding fusion feature, wherein the first feature fusion model is a lightweight feature fusion model; a second feature fusion model, configured to perform weighted fusion based on the signal quality score corresponding to the target decoded signal and the fusion feature to obtain a corresponding fusion feature, wherein the first feature fusion model is a deep learning model; The priority judgment unit is used to obtain the decoding requirements corresponding to the current decoding task, and determine whether the first feature fusion model or the second feature fusion model is used to perform feature fusion work according to the decoding requirements.

[0100] As for the above-mentioned device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0101] Embodiment 8: A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of the above embodiments 1 to 4 is implemented.

[0102] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0103] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, apparatus, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0104] The present invention is described with reference to the flowcharts and / or block diagrams of the method, terminal device (system), and computer program product according to the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the process in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0105] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0106] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0107] It should be noted that: References in this specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Therefore, appearances of the phrases "one embodiment" or "an embodiment" in various places throughout this specification do not necessarily refer to the same embodiment.

[0108] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0109] Furthermore, it should be noted that the specific embodiments described in this specification may vary in the shapes and names of their components. Any equivalent or simple variations based on the structure, features, and principles described in the patented concept of this invention are included within the scope of protection of this patent. Persons skilled in the art may make various modifications, additions, or substitutions to the described specific embodiments, and these modifications, as long as they do not deviate from the structure of the invention or exceed the scope defined by the claims, shall fall within the scope of protection of this invention.

Claims

1. A motion intention decoding method, characterized in that: The following steps are involved: Acquire first signal data, where the first signal data includes one or more physiological signals corresponding to a current exercise period of the user; Based on the first signal data and the prediction model, a corresponding prediction result is generated, where the prediction result is used to indicate the movement intention corresponding to the user's next action; Based on the first signal data and the intention recognition model, a corresponding decoding result is generated and output, wherein the decoding result is used to indicate the movement intention corresponding to the user's current action; Calculating the degree of deviation between the decoding result and the prediction result; When the degree of deviation exceeds a preset deviation threshold, the intention recognition model is reversely updated based on the degree of deviation.

2. The motion intention decoding method according to claim 1, characterized in that: The first signal data includes a first physiological signal and a plurality of second physiological signals, wherein the first physiological signal is an electroencephalogram signal; The intention recognition model includes a unimodal decoding model and a multimodal decoding model. When the deviation between the corresponding decoding result and the prediction result exceeds a preset deviation threshold, the unimodal decoding model and the multimodal decoding model are reversely updated based on the deviation; Based on the first signal data and the intention recognition model, the specific steps of generating a corresponding decoding result include: Comparing the signal quality score corresponding to the first physiological signal with a preset first score threshold; When the signal quality score corresponding to the first physiological signal reaches the first score threshold, decoding the first physiological signal based on the single-modality decoding model to obtain a corresponding decoding result; When the signal quality score corresponding to the first physiological signal does not reach the first scoring threshold, the first physiological signal is used as the target decoding signal, and the second physiological signal with a signal quality score higher than the second scoring threshold is used as the target decoding signal. The multimodal decoding model performs multimodal fusion decoding based on each target decoding signal to obtain a corresponding decoding result.

3. The motion intention decoding method according to claim 2, characterized in that: The second physiological signal is an electromyographic signal, a functional near infrared signal and a posture signal.

4. The motion intention decoding method according to claim 3, characterized in that: The prediction model is a digital twin prediction model; The input of the digital twin prediction model is EEG signals, EMG signals and posture signals, and the output is the corresponding prediction results.

5. The motion intention decoding method according to any one of claims 2 to 4, characterized in that: The specific steps of performing multimodal fusion decoding based on each target decoding signal by the multimodal decoding model are as follows: Extracting decoding features corresponding to each target decoding signal; Perform feature weighting processing based on the signal quality score and decoding features corresponding to each target decoding signal to obtain corresponding fusion features; The fusion feature is input into the multimodal decoding model, and the multimodal decoding model outputs a corresponding decoding result.

6. The motion intention decoding method according to claim 5, characterized in that: Based on the signal quality score and decoding features corresponding to each target decoded signal, feature weighting processing is performed to obtain the corresponding fusion features in the following specific steps: Get the decoding requirements corresponding to the current decoding task; When the decoding requirement is real-time, a first feature fusion model performs weighted fusion based on the signal quality score corresponding to the target decoding signal and the fusion feature to obtain a corresponding fusion feature, wherein the first feature fusion model is a lightweight feature fusion model; When the decoding requirement is accuracy, the second feature fusion model performs weighted fusion based on the signal quality score and fusion features corresponding to the target decoding signal to obtain the corresponding fusion features. The second feature fusion model is a deep learning model.

7. The motion intention decoding method according to claim 2, characterized in that: The method for calculating the signal quality score corresponding to the first physiological signal is: A corresponding signal quality score is calculated based on the signal-to-noise ratio, spectral entropy and power ratio of the first physiological signal.

8. The motion intention decoding method according to claim 3, characterized in that: The calculation method of the signal quality score corresponding to each second physiological signal includes: Based on the root mean square value, median frequency of the electromyographic spectrum and waveform complexity of the electromyographic signal, the corresponding signal quality score is calculated; Based on the signal amplitude change, optical signal intensity and spectrum characteristics corresponding to the functional near-infrared signal, the corresponding signal quality score is calculated; Based on the acceleration mean deviation, angular velocity stability and signal sampling rate stability corresponding to the attitude signal, the corresponding signal quality score is calculated.

9. A motion intention decoding system, characterized in that: include: An acquisition module, configured to acquire first signal data, wherein the first signal data includes one or more physiological signals corresponding to a current exercise period of the user; A prediction system, used to generate a corresponding prediction result based on the first signal data and the prediction model, wherein the prediction result is used to indicate the movement intention corresponding to the user's next action; A decoding system, used to generate and output a corresponding decoding result based on the first signal data and the intention recognition model, wherein the decoding result is used to indicate the movement intention corresponding to the user's current action; The feedback system is used to calculate the degree of deviation between the decoding result and the prediction result, and is also used to reversely update the intention recognition model based on the degree of deviation when the degree of deviation exceeds a preset deviation threshold.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images