A surface myoelectric signal recognition method, model training method and system

Abstract

The application provides a surface electromyogram signal recognition method, a model training method and a system. The method comprises the following steps: training an initial surface electromyogram signal recognition model through a training sample set comprising a plurality of two-dimensional time-frequency spectrograms; performing local time-frequency feature extraction by a local adjustment extraction module, converting by a feature sequence reconstruction module, and superimposing a spatial bias by a spatial position coding module; inputting a global space-time fusion feature sequence output by a space-time context modeling module; finally outputting an action category prediction value, a joint state prediction value and a muscle fatigue index prediction value by a multi-task parallel output module; constructing a fusion loss function with the deviation of the label to update the parameters of the initial surface electromyogram signal recognition model to obtain a surface electromyogram signal recognition model; and realizing the stability and accuracy of surface electromyogram signal recognition and real-time feedback of muscle state.

Description

Technical Field

[0001] This invention relates to the field of surface electromyography (EMG) signal recognition, and more particularly to a surface EMG signal recognition method, model training method, and system. Background Technology

[0002] Surface electromyography (sEMG) can collect changes in bioelectric potential generated by muscle contraction in the human body, analyze the user's movement intention, quantify the degree of muscle activation and fatigue state, and is widely used in post-stroke hand function rehabilitation, limb movement assessment and human-computer interaction scenarios. However, existing recognition schemes have multiple inherent defects and are difficult to meet the requirements of accurate, safe and private use in clinical rehabilitation training.

[0003] First, the quality of electromyography (EMG) signal acquisition depends on the hardware contact status and the individual physiological conditions of the subject. The signal stability is poor and it is easily affected by fluctuations in skin impedance, sweating, and electrode displacement. As the training time increases, the accuracy of gesture and movement recognition continues to decline, and the distribution of EMG signals collected from different subjects shows significant deviations, indicating insufficient signal robustness. Second, the existing deep learning models for EMG recognition have weak generalization ability and are difficult to adapt to diverse hand movement training scenarios. Third, the existing recognition system relies solely on rehabilitation physicians to manually judge the standard of movement, and cannot quantify and display the activation intensity of each muscle group in real time. It also lacks real-time muscle fatigue monitoring methods and cannot dynamically assess the muscle fatigue index. Finally, the existing solution requires uploading all data collected by the terminal during the training process to the cloud to complete unified model training, which does not comply with medical data security compliance requirements.

[0004] Therefore, to address the shortcomings of existing technologies, a surface electromyography (EMG) signal recognition method that balances robust signal recognition, personalized adaptation, and real-time rehabilitation feedback is needed. Summary of the Invention

[0005] This invention provides a surface electromyography (EMG) signal recognition method, model training method, and system. Its main purpose is to solve the problems of poor stability, poor versatility, and inability to provide real-time feedback on muscle status in existing technologies for surface EMG signal recognition.

[0006] To achieve the above objectives, the present invention provides a training method for a surface electromyography signal recognition model, the training method comprising the following steps: A training sample set is obtained, which contains multiple samples, each of which contains a two-dimensional time-frequency spectrogram. Each sample is labeled with the true value of the action category, the true value of the joint state, and the true value of the muscle fatigue index. The two-dimensional time-frequency spectrogram is obtained by performing time-frequency transformation on the surface electromyography signals of the limb to be tested. An initial surface electromyography (SEMG) signal recognition model is obtained, comprising a time-spectrum input module, a local adjustment extraction module, a feature sequence reconstruction module, a spatial location encoding module, a spatiotemporal context modeling module, and a multi-task parallel output module. The time-spectrum input module inputs the two-dimensional time-spectrum into the local adjustment extraction module for local time-frequency feature extraction, outputting a three-dimensional feature tensor. This tensor is then converted into a feature sequence after dimensional mapping by the feature sequence reconstruction module, and outputs a feature sequence with superimposed spatial bias through the spatial location encoding module. The spatiotemporal context modeling module comprises a multi-head self-attention layer, a feedforward fully connected layer, layer normalization, and a stacked layer of residual connections. It takes the feature sequence with superimposed spatial bias as input and outputs a global spatiotemporal fusion feature sequence. The multi-task parallel output module comprises an action classification branch, a motion state branch, and a muscle fatigue assessment branch. It takes the global spatiotemporal fusion feature sequence as input. The action classification branch outputs predicted action categories, the motion state branch outputs predicted joint states, and the muscle fatigue assessment branch outputs predicted muscle fatigue indices. A first loss function is constructed based on the deviation between the actual value and the predicted value of the action category. A second loss function is constructed based on the deviation between the actual value and the predicted value of the joint state. A third loss function is constructed based on the deviation between the actual value and the predicted value of the muscle fatigue index. The first, second, and third loss functions are weighted and summed to obtain a fusion loss function. With the goal of minimizing the fusion loss, the parameters of the initial surface electromyography (SEMG) signal recognition model are updated using the training sample set until convergence is obtained to obtain the SEMG signal recognition model.

[0007] Optionally, the method further includes: Each terminal receives the surface electromyography (EMG) signal recognition model sent by the cloud server and collects the EMG signals of the limb under test when it performs a preset calibration action. The parameters of the multi-head self-attention layer of the spatiotemporal context modeling module and the multi-task parallel output module are adjusted and adapted to the target EMG signal recognition model of each terminal. Each terminal uploads the gradient parameters generated by the surface electromyography signal recognition model training to the cloud server, and the gradient parameters uploaded by each terminal are aggregated by the federated averaging algorithm and then distributed to each terminal device.

[0008] Optionally, the action classification branch includes two fully connected layers and a Softmax activation layer, the motion state branch includes three fully connected layers, and the muscle fatigue assessment branch includes two fully connected layers.

[0009] Optionally, the two-dimensional time-frequency spectrum is obtained by performing time-frequency transformation on the surface electromyography signals of the limb under test, including: Acquire surface electromyography (EMG) signals of the limb under test from multiple channels, and extract multi-segmented signals from the surface EMG signals using a sliding window. Standardized signals are obtained by filtering, baseline drift correction, and normalization of the segmented signals of each group; The standardized signal is converted into the two-dimensional time-spectrum diagram by short-time Fourier transform.

[0010] Optionally, the method further includes: A digital twin 3D model of the limb to be tested is constructed based on the test scenario and the preset behaviors and actions within the scenario; The two-dimensional time-frequency spectrum obtained by time-frequency transformation of the surface electromyography signal of the limb under test is input into the pre-trained surface electromyography signal recognition model to obtain the predicted values ​​of action category, joint state and muscle fatigue index. Combined with the preset control logic, a control scheme for coordinating the limb under test is generated. The control scheme is simulated on the digital twin 3D model. The control scheme is adjusted according to the simulation results. The adjusted control scheme is then applied to the limb under test and its status is displayed.

[0011] Optionally, the method further includes: real-time monitoring of the activation levels of the shoulder muscles and forearm muscles of the limb under test; outputting a functional electrical stimulation pulse of a preset frequency to the limb under test when the activation levels of the shoulder muscles and forearm muscles are lower than a preset threshold; and outputting a compensation prompt and pausing the current action when the activation levels of the shoulder muscles are abnormal and the activation levels of the forearm muscles exhibit compensatory movements.

[0012] Optionally, the three-dimensional feature tensor includes a sample batch dimension, a total number of electromyographic electrode channels, and a single-channel feature dimension.

[0013] Optionally, the method further includes: the local adjustment extraction module employs a residual convolutional network.

[0014] To address the above problems, the present invention also provides a method for identifying surface electromyography signals, the method comprising the following steps: Acquire surface electromyographic signals of the limb to be tested; The surface electromyography (EMG) signal is input into the surface EMG signal recognition model in any of the above-mentioned surface EMG signal recognition model training methods, and the action category, joint state, and muscle fatigue index of the surface EMG signal of the limb under test are output.

[0015] To address the aforementioned problems, the present invention also provides a surface electromyography signal recognition system, comprising a processor, a memory, and a computer program / instructions stored in the memory, characterized in that the processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the system implements the steps of any of the methods described above.

[0016] The beneficial effects of this invention are as follows: This invention provides a surface electromyography (SEMG) signal recognition method, model training method, and system. The SEMG signal recognition model includes a time-spectrum input module, a local adjustment extraction module, a feature sequence reconstruction module, a spatial location encoding module, a spatiotemporal context modeling module, and a multi-task parallel output module, which improves the accuracy of SEMG signal recognition. It utilizes digital twin technology for real-time visualization of muscle status, identifies compensatory movements and muscle fatigue, and effectively prevents muscle damage caused by overtraining. It adopts a federated learning architecture to protect patient privacy.

[0017] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the description, or may be learned by practice of the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures specifically pointed out in the description and drawings.

[0018] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0019] Figure 1 This is a schematic flowchart of a surface electromyography signal recognition model training method according to an embodiment of the present invention.

[0020] Figure 2 This is a schematic diagram illustrating the time-frequency transformation of the surface electromyography signal of the limb to be tested according to an embodiment of the present invention.

[0021] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0023] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0024] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0025] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0026] In the following description, embodiments of the invention will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0027] Figure 1 This is a flowchart illustrating a surface electromyography (EMG) signal recognition model training method according to an embodiment of the present invention. This application provides a surface EMG signal recognition model training method, the process of which includes the following steps S101~S103: Step S101: Obtain the training sample set, which contains multiple samples. Each sample contains a two-dimensional time-frequency spectrum. Each sample is labeled with the true value of the action category, the true value of the joint state, and the true value of the muscle fatigue index. The two-dimensional time-frequency spectrum is obtained by performing time-frequency transformation on the surface electromyography signals of the limb to be tested.

[0028] Step S102: Obtain the initial surface electromyography (EMG) signal recognition model. The initial EMG signal recognition model includes a time-spectrum input module, a local adjustment extraction module, a feature sequence reconstruction module, a spatial location encoding module, a spatiotemporal context modeling module, and a multi-task parallel output module. The time-spectrum input module inputs a two-dimensional time-spectrum image into the local adjustment extraction module for local time-frequency feature extraction, outputting a three-dimensional feature tensor. After dimensional mapping by the feature sequence reconstruction module, it is converted into a feature sequence, and the spatial location encoding module outputs a feature sequence with superimposed spatial bias. The spatiotemporal context modeling module includes a multi-head self-attention layer, a feedforward fully connected layer, layer normalization, and residual connection stacked layers. The spatiotemporal context modeling module takes the feature sequence with superimposed spatial bias as input and outputs a global spatiotemporal fusion feature sequence. The multi-task parallel output module includes an action classification branch, a motion state branch, and a muscle fatigue assessment branch. The multi-task parallel output module takes the global spatiotemporal fusion feature sequence as input. The action classification branch outputs the action category prediction value, the motion state branch outputs the joint state prediction value, and the muscle fatigue assessment branch outputs the muscle fatigue index prediction value.

[0029] Step S103: Construct a first loss function based on the deviation between the actual value and the predicted value of the action category; construct a second loss function based on the deviation between the actual value and the predicted value of the joint state; construct a third loss function based on the deviation between the actual value and the predicted value of the muscle fatigue index; calculate the fusion loss function by weighted summation of the first, second, and third loss functions; update the parameters of the initial surface electromyography (SEMG) signal recognition model using the training sample set with the goal of minimizing the fusion loss, until convergence is obtained to obtain the SEMG signal recognition model.

[0030] In step S101, the surface electromyography signal of the limb to be tested is acquired through a multi-channel high-density flexible array electrode. The flexible electrode scheme can improve the adhesion between the electrode and the skin and reduce contact noise. Figure 2 This is a schematic flowchart illustrating the time-frequency transformation of the surface electromyography (EMG) signal of the limb under test according to an embodiment of the present invention. In this embodiment, the process of obtaining the two-dimensional time-frequency spectrum by performing time-frequency transformation on the acquired surface EMG signal of the limb under test includes steps S1011~S1013: Step S1011: Acquire the surface electromyography (EMG) signals of the limb under test from multiple channels, and use a sliding window to extract multiple segments of the surface EMG signals.

[0031] Step S1012: Filter, correct baseline drift, and normalize the signals of each component segment to obtain a standardized signal.

[0032] Step S1013: Convert the standardized signal into a two-dimensional time spectrum using short-time Fourier transform.

[0033] Specifically, multiple sets of segmented signals to be processed are acquired according to the preset sliding window time length and sliding step size; filtering processing includes bandpass filtering and notch filtering, with notch filtering removing power supply noise commonly present in the signal acquisition environment; for baseline drift caused by patient breathing, sweating, or relative displacement between the electrode and the skin, the moving average method is used for correction; the standardized signal is a one-dimensional time-series signal.

[0034] In step S102, in this embodiment of the invention, the method further includes: a local adjustment extraction module employing a residual convolutional network (CNN). The CNN uses ResNet18 to extract local features, and each electrode channel outputs a set of independent high-dimensional feature vectors, which are combined to form a multi-channel feature sequence, which is then fed into the subsequent feature sequence reconstruction module; the spatiotemporal context modeling module uses Transformer to extract long sequence dependencies; in this embodiment of the invention, the action classification branch includes two fully connected layers and a Softmax activation layer, the motion state branch includes three fully connected layers, and the muscle fatigue assessment branch includes two fully connected layers. In this embodiment of the invention, the three-dimensional feature tensor includes the sample batch dimension, the total number of electromyographic electrode channels, and the single-channel feature dimension.

[0035] Furthermore, the feature sequence reconstruction module performs a unified dimensional mapping of the three-dimensional feature tensor into a feature sequence. The feature sequence reconstruction module internally sets up a linear projection layer to uniformly convert the three-dimensional feature tensor into a dimension that can be used later. The feature sequence is input to the spatial position encoding module, which constructs a spatial bias based on the actual physical distance of each electromyography (EMG) electrode in the limb to be tested, and superimposes the spatial bias onto the feature sequence, outputting a feature sequence carrying the spatial bias. The spatiotemporal context modeling module performs multi-head self-attention weight calculation based on the feature sequence carrying the spatial bias. During the calculation, the spatial bias corresponding to the physical position of the electrode is introduced to correct the attention score, thereby improving the initial attention association weight between electrode channels with adjacent physical positions.

[0036] In step S103, the first loss function, the second loss function, and the third loss function include, but are not limited to, the multi-class cross-entropy loss function and the mean squared error loss function.

[0037] In this embodiment of the invention, the method further includes: Each terminal receives the surface electromyography (SEMG) signal recognition model from the cloud server and collects SEMG signals from the limb being tested when it performs a preset calibration movement. The parameters of the multi-head self-attention layer of the spatiotemporal context modeling module and the multi-task parallel output module are adjusted and adapted to the target SEMG signal recognition model for each terminal. Each terminal uploads the gradient parameters generated during the training of the SEMG signal recognition model to the cloud server, and the federated averaging algorithm aggregates the gradient parameters uploaded by each terminal before distributing them to each terminal device.

[0038] Specifically, each terminal uses local electromyography data to calculate the model gradient independently and uploads the gradient to the cloud. The cloud performs a weighted average aggregation of the gradient parameters to obtain the global gradient. After correcting the global model weights stored on the cloud server with the global gradient, the new model is distributed back to the terminal.

[0039] In this embodiment of the invention, the method further includes: A digital twin 3D model of the limb to be tested is constructed based on the test scenario and the preset behaviors and actions within the scenario.

[0040] The two-dimensional time-frequency spectrum obtained by time-frequency transformation of the surface electromyography (EMG) signal of the limb under test is input into the pre-trained EMG signal recognition model to obtain the predicted values ​​of action category, joint state, and muscle fatigue index. Combined with the preset control logic, a control scheme for the limb under test is generated.

[0041] Specifically, the Muscle Fatigue Index (MFI) is obtained by real-time detection of the ratio of the rate of decrease in the frequency domain characteristics to the rate of increase in the time domain characteristics of surface electromyography signals.

[0042] The control scheme is simulated and run on a digital twin 3D model. Based on the simulation results, the control scheme is adjusted, and the adjusted control scheme is applied to the tested limb, displaying its status. Specifically, the digital twin 3D model maps surface muscle activation heatmaps in real time, and dynamically adjusts the task difficulty in the control scheme or issues rest commands based on the muscle fatigue index.

[0043] In this embodiment of the invention, the method further includes: real-time monitoring of the activation levels of the shoulder muscles and forearm muscles of the limb under test; outputting a functional electrical stimulation pulse of a preset frequency to the limb under test when the activation levels of the shoulder muscles and forearm muscles are lower than a preset threshold; and outputting a compensation prompt message and pausing the current action when the activation levels of the shoulder muscles are abnormal and there is a compensatory movement in the activation levels of the forearm muscles.

[0044] On the other hand, the present invention also provides a method for identifying surface electromyography signals, the method comprising the following steps: Acquire surface electromyographic signals of the limb to be tested; The surface electromyography (EMG) signal is input into the surface EMG signal recognition model as described in the above-mentioned surface EMG signal recognition model training method, and the action category, joint state, and muscle fatigue index of the surface EMG signal of the limb to be tested are output.

[0045] On the other hand, the present invention also provides a surface electromyography signal recognition system, including a processor, a memory, and a computer program / instructions stored in the memory, characterized in that the processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the system implements the steps of the above method.

[0046] The present invention will now be described with reference to a specific embodiment: This invention proposes a surface electromyography (EMG) signal recognition method, a model training method, and a system. The EMG signal recognition system includes a signal acquisition module, a signal processing module, a recognition module, a feedback training module, and a distributed training module.

[0047] (1) Signal acquisition module: acquires multi-channel surface electromyography signals through flexible electrode array; the 8-16 channels of the flexible electrode acquire surface electromyography voltage signals. The flexible electrode array adopts 8-channel flexible electrodes based on 3D printing technology. The electrodes are arranged on the main muscle groups of the user's forearm, such as the flexor carpi ulnaris and extensor carpi radialis longus, according to the human anatomy. The electrode spacing is set to 2cm to cover the activation area of ​​a single muscle. The analog signals acquired by the electrodes are differentially amplified by the instrumentation amplifier, with the gain set to 1000 times, and then digitized by a 16-bit ADC at a sampling rate of 1000Hz.

[0048] (2) Signal processing module: preprocesses the received data, removes power frequency interference by bandpass filtering and Notch filtering, removes low frequency drift caused by the relative displacement between the electrode and the skin by moving average method, generates a standardized signal by normalization, and converts the one-dimensional time sequence signal into a two-dimensional time spectrum by time-frequency transformation.

[0049] (3) Recognition module: The standardized signal is input into the surface electromyography signal recognition model and the output includes the recognition result including the action category, joint state and muscle activation. The surface electromyography signal recognition model includes a convolutional neural network submodule and a temporal attention submodule. The convolutional neural network submodule (CNN) adopts the ResNet18 structure, the input is the two-dimensional time spectrum of each channel, and the output is a high-dimensional feature vector. The temporal attention submodule (Transformer) adds position encoding to preserve the spatial position relationship of the channels. The multi-head self-attention mechanism is used to calculate the co-activation weights between different muscle groups. The convolutional neural network submodule is used to extract the local time-frequency features of the signal, and the temporal attention submodule is used to capture the long-range dependencies between different channels. Spatial location coding based on the physical distribution of electrodes is introduced. Spatial location coding is achieved by constructing an adjacency matrix. The element values ​​in the adjacency matrix are weighted according to the physical distance of the electrodes on the flexible substrate or the anatomical position of the human body, so that the electrode channels that are physically adjacent have higher correlation weights in the attention calculation. The output head includes a classification head, a regression head, and a fatigue head. The classification head outputs discrete action probabilities of clenching fist, extending palm, and pinching. The regression head outputs continuous values ​​of joint state and force percentage. The fatigue head predicts the muscle fatigue level based on the median frequency (MDF) decreasing trend.

[0050] (4) Feedback training module: Based on the recognition results, virtual training tasks are generated, and the difficulty of the tasks or physical feedback stimuli are dynamically adjusted according to the user's performance. The feedback training module calculates the muscle fatigue index (MFI) based on the ratio of the change rate of the time domain feature to the change rate of the frequency domain feature of the electromyographic signal within a preset time window. When the muscle fatigue index exceeds the preset threshold, the feedback training module pauses the virtual training task and issues a rest prompt. The physical feedback stimuli output by the feedback training module include functional electrical stimulation (FES) or vibration feedback, and the intensity and timing of the stimulation are synchronized with the action requirements in the virtual training task. A digital twin model of the user's limbs is constructed in the virtual space, and a muscle activation heat map is mapped in real time. The difficulty of the VR training task is dynamically adjusted or a rest instruction is issued according to the recognized muscle fatigue index.

[0051] In a stroke patient hand rehabilitation training scenario, a virtual orchard scene is displayed where the patient is picking apples. The system recognizes the patient's electromyographic signal indicating an intention to extend their hand and displays a digital twin model of the patient's forearm on the side of the screen, rendering a muscle heat map in real time. Red indicates high activation and blue indicates low activation in the muscle heat map. When the muscle activation level is detected to be 30%, the virtual apple cannot be picked. The electrical stimulator outputs 20Hz functional electrical stimulation to assist muscle contraction. When abnormally active deltoid muscle signals are detected, while the target forearm signal is weak, it is determined to be compensation, and the system immediately pops up a prompt: Please relax your shoulder and pause the task.

[0052] The system monitors the muscle fatigue index in real time. When it detects that the median frequency has dropped by more than 15% and the RMS amplitude has increased by more than 20% (i.e., MFI>0.8), the system automatically ends the training and suggests rest to prevent muscle strain.

[0053] (5) Distributed training module, which executes federated learning protocol; trains the parameters of surface electromyography signal recognition model locally on multiple terminal devices, uploads only the parameter gradients to the cloud server for aggregation, and sends the updated global model to the terminal after aggregation, without transmitting the original electromyography data; the distributed training module is also configured with a personalized adaptation layer, which retains the specific network layer parameters of each terminal device after aggregation in the cloud, and only updates the shared feature extraction layer parameters to adapt to the differences in electromyography patterns of different users.

[0054] Users perform a 5-minute calibration process upon first use, during which the model is fine-tuned locally to adapt to individual electromyography patterns. Multiple users' edge devices upload only the model's gradient parameters to the cloud server without uploading raw data. The cloud aggregates the global model using a federated averaging algorithm (FedAvg) and distributes the updated model to each terminal. Personalized federated learning (pFed) is introduced to preserve users' individual characteristics on top of the global model.

[0055] The present invention also provides a method for recognizing surface electromyography (EMG) signals, which is performed on the above-described surface EMG signal recognition system: The raw surface electromyography signal is received, bandpass filtered to remove power frequency interference and baseline drift, and full-wave rectification and moving average filtering are performed to obtain the signal envelope.

[0056] The preprocessed signal was subjected to a short-time Fourier transform (STFT) with a Hanning window function, a window length of 256 ms, and an overlap rate of 50%. The generated time spectrum was normalized to the [0,1] interval and used as the input tensor of the model.

[0057] The time-frequency spectrogram is input into the CNN-Transformer hybrid neural network model. Local features are extracted by the convolutional neural network, and the spatiotemporal dependencies between multiple channels are captured by the Transformer encoder. The signal is then fed into the first convolutional layer of the ResNet-18 network. After passing through three residual blocks, a high-dimensional feature vector is output. Here, the translation invariance of the convolutional neural network submodule (CNN) is used to capture the local time-frequency features of the electromyographic signal. The spatial location encoding of the Transformer spatiotemporal model constructs the electrode adjacency matrix. During learning, the model prioritizes the relationship between physically adjacent or functionally coordinated muscle groups. After multi-head attention, the global context features are output.

[0058] The multi-task output includes a classification head, a regression head, and a fatigue head. The classification head is activated by Softmax and outputs the probability of 10 types of gestures; the regression head is activated by linearity and outputs the wrist flexion angle; the fatigue head is activated by Sigmoid and outputs the muscle fatigue index.

[0059] In summary, this invention provides a surface electromyography (SEMG) signal recognition method, model training method, and system. The method involves acquiring a training sample set containing multiple samples, each containing a two-dimensional time-frequency spectrogram. Each sample is labeled with the true values ​​of the action category, joint state, and muscle fatigue index. The two-dimensional time-frequency spectrogram is obtained by performing time-frequency transformation on the collected SEMG signals of the limb under test. An initial SEMG signal recognition model is then acquired, comprising a time-frequency spectrogram input module, a local adjustment extraction module, a feature sequence reconstruction module, a spatial location encoding module, a spatiotemporal context modeling module, and a multi-task parallel output module. The time-frequency spectrogram input module inputs the two-dimensional time-frequency spectrogram into the local adjustment extraction module for local time-frequency feature extraction, outputting a three-dimensional feature tensor. This tensor is then converted into a feature sequence after dimensional mapping by the feature sequence reconstruction module, and finally output by the spatial location encoding module with superimposed spatial bias. The feature sequence and spatiotemporal context modeling module take the feature sequence with superimposed spatial bias as input and output a global spatiotemporal fusion feature sequence. The multi-task parallel output module includes an action classification branch, a motion state branch, and a muscle fatigue assessment branch. It takes the global spatiotemporal fusion feature sequence as input and outputs the action category prediction value, joint state prediction value, and muscle fatigue index prediction value. A first loss function is constructed based on the deviation between the actual value and the predicted value of the action category. A second loss function is constructed based on the deviation between the actual value and the predicted value of the joint state. A third loss function is constructed based on the deviation between the actual value and the predicted value of the muscle fatigue index. The first loss function, the second loss function, and the third loss function are weighted and summed to obtain the fusion loss function. With the goal of minimizing the fusion loss, the parameters of the initial surface electromyography (EMG) signal recognition model are updated using the training sample set until convergence is obtained.

[0060] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.

[0061] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0062] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0063] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for training a surface electromyography (EMG) signal recognition model, characterized in that, The training method involves the following steps: A training sample set is obtained, which contains multiple samples, each of which contains a two-dimensional time-frequency spectrogram. Each sample is labeled with the true value of the action category, the true value of the joint state, and the true value of the muscle fatigue index. The two-dimensional time-frequency spectrogram is obtained by performing time-frequency transformation on the surface electromyography signals of the limb to be tested. An initial surface electromyography (SEMG) signal recognition model is obtained, comprising a time-spectrum input module, a local adjustment extraction module, a feature sequence reconstruction module, a spatial location encoding module, a spatiotemporal context modeling module, and a multi-task parallel output module. The time-spectrum input module inputs the two-dimensional time-spectrum into the local adjustment extraction module for local time-frequency feature extraction, outputting a three-dimensional feature tensor. This tensor is then converted into a feature sequence after dimensional mapping by the feature sequence reconstruction module, and outputs a feature sequence with superimposed spatial bias through the spatial location encoding module. The spatiotemporal context modeling module comprises a multi-head self-attention layer, a feedforward fully connected layer, layer normalization, and a stacked layer of residual connections. It takes the feature sequence with superimposed spatial bias as input and outputs a global spatiotemporal fusion feature sequence. The multi-task parallel output module comprises an action classification branch, a motion state branch, and a muscle fatigue assessment branch. It takes the global spatiotemporal fusion feature sequence as input. The action classification branch outputs predicted action categories, the motion state branch outputs predicted joint states, and the muscle fatigue assessment branch outputs predicted muscle fatigue indices. A first loss function is constructed based on the deviation between the actual value and the predicted value of the action category. A second loss function is constructed based on the deviation between the actual value and the predicted value of the joint state. A third loss function is constructed based on the deviation between the actual value and the predicted value of the muscle fatigue index. The first, second, and third loss functions are weighted and summed to obtain a fusion loss function. With the goal of minimizing the fusion loss, the parameters of the initial surface electromyography (SEMG) signal recognition model are updated using the training sample set until convergence is obtained to obtain the SEMG signal recognition model.

2. The surface electromyography signal recognition model training method according to claim 1, characterized in that, The method also includes: Each terminal receives the surface electromyography (EMG) signal recognition model sent by the cloud server and collects the EMG signals of the limb under test when it performs a preset calibration action. The parameters of the multi-head self-attention layer of the spatiotemporal context modeling module and the multi-task parallel output module are adjusted and adapted to the target EMG signal recognition model of each terminal. Each terminal uploads the gradient parameters generated by the surface electromyography signal recognition model training to the cloud server, and the gradient parameters uploaded by each terminal are aggregated by the federated averaging algorithm and then distributed to each terminal device.

3. The surface electromyography signal recognition model training method according to claim 1, characterized in that, The action classification branch includes two fully connected layers and a Softmax activation layer, the motion state branch includes three fully connected layers, and the muscle fatigue assessment branch includes two fully connected layers.

4. The surface electromyography signal recognition model training method according to claim 1, characterized in that, The process of obtaining the two-dimensional time-frequency spectrum by performing time-frequency transformation on the surface electromyography signals of the limb under test includes: Acquire surface electromyography (EMG) signals of the limb under test from multiple channels, and extract multi-segmented signals from the surface EMG signals using a sliding window. Standardized signals are obtained by filtering, baseline drift correction, and normalization of the segmented signals of each group; The standardized signal is converted into the two-dimensional time-spectrum diagram by short-time Fourier transform.

5. The surface electromyography signal recognition model training method according to claim 1, characterized in that, The method also includes: A digital twin 3D model of the limb to be tested is constructed based on the test scenario and the preset behaviors and actions within the scenario; The two-dimensional time-frequency spectrum obtained by time-frequency transformation of the surface electromyography signal of the limb under test is input into the pre-trained surface electromyography signal recognition model to obtain the predicted values ​​of action category, joint state and muscle fatigue index. Combined with the preset control logic, a control scheme for the limb under test is generated. The control scheme is simulated and run on the digital twin 3D model. The control scheme is adjusted according to the simulation results. The adjusted control scheme is then applied to the limb under test and its status is displayed.

6. The surface electromyography signal recognition model training method according to claim 1, characterized in that, The method further includes: real-time monitoring of the activation levels of the shoulder muscles and forearm muscles of the limb under test; outputting a functional electrical stimulation pulse of a preset frequency to the limb under test when the activation levels of the shoulder muscles and forearm muscles are lower than a preset threshold; and outputting a compensation prompt and pausing the current action when the activation levels of the shoulder muscles are abnormal and the activation levels of the forearm muscles are compensating.

7. The surface electromyography signal recognition model training method according to claim 1, characterized in that, The three-dimensional feature tensor includes sample batch dimension, total number of electromyographic electrode channels, and single-channel feature dimension.

8. The surface electromyography signal recognition model training method according to claim 1, characterized in that, The method further includes: the local adjustment extraction module employs a residual convolutional network.

9. A method for recognizing surface electromyography signals, characterized in that, The method includes the following steps: Acquire surface electromyographic signals of the limb to be tested; The surface electromyography (EMG) signal is input into the surface EMG signal recognition model in the surface EMG signal recognition model training method as described in any one of claims 1 to 8, and the action category, joint state, and muscle fatigue index of the surface EMG signal of the limb under test are output.

10. A surface electromyography signal recognition system, characterized in that, The system includes a processor, a memory, and a computer program / instructions stored in the memory, characterized in that the processor is configured to execute the computer program / instructions, and when the computer program / instructions are executed, the system implements the steps of the method as described in any one of claims 1 to 9.

Images