A Virtual Keyboard Keystroke Gesture Recognition Method Based on Electromyographic Signals and an Electromyographic Armband

Abstract

This invention discloses a virtual keyboard key gesture recognition method and an electromyography (EMG) armband based on electromyography (EMG) signals. During the training phase, the method simultaneously acquires forearm surface EMG signals and finger joint angle signals during the subject's key presses. The two modal signals undergo cascaded preprocessing and are then input into a cross-modal knowledge distillation training process. By using the joint model as the teacher model and the EMG model as the student model, knowledge transfer from the joint model to the EMG model is achieved, thereby improving the gesture recognition accuracy of the joint model. Finally, gesture recognition is performed. The recognition device employs two EMG signal acquisition modes: alternating acquisition of adjacent channels and joint acquisition. This invention significantly improves the accuracy of the EMG model in recognizing similar key presses through a cross-modal knowledge distillation strategy. Furthermore, by relying solely on the EMG model for recognition during the inference phase, the system's wearability, real-time performance, and environmental adaptability are enhanced, making it applicable to scenarios where traditional keyboards are inconvenient to use.

Description

Technical Field

[0001] This invention relates to the fields of human-computer interaction and biosignal processing technology, and in particular to a virtual keyboard key gesture recognition method and electromyographic armband based on electromyographic signals. Background Technology

[0002] In scenarios where physical keyboards are unavailable, such as driving, industrial operations, and medical rehabilitation, virtual numeric keyboards have become an important text input method. Existing methods include speech recognition, eye tracking, and electroencephalography (EEG), but these suffer from problems such as significant environmental interference, high false triggering rates, and poor wearability. Surface electromyography (EMG), due to its strong anti-interference capabilities, high motion resolution, and good wearability, is gradually becoming a research hotspot.

[0003] However, electromyographic signals still suffer from insufficient accuracy when recognizing similar key gestures. While multimodal fusion can improve recognition performance, it increases system complexity. Traditional electromyographic armbands often use fixed multi-channel synchronous acquisition, lacking adaptive adjustments for movement accuracy and muscle size. This can easily lead to information redundancy in large muscle group tasks and affect discrimination ability in fine-grained recognition of small muscle groups.

[0004] Therefore, how to improve the recognition accuracy and optimize the acquisition performance of the electromyography armband while maintaining the ease of wearing the device is an urgent problem to be solved. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the purpose of this invention is to provide a virtual keyboard key gesture recognition method and an electromyographic armband based on electromyographic signals, which aims to solve the problem of insufficient accuracy of electromyographic signals in recognizing similar key gestures in existing virtual keyboard key gesture recognition methods.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] Firstly, a virtual keyboard key gesture recognition method based on electromyographic signals, the key gesture recognition method including:

[0008] Step 1: Synchronously collect electromyographic signals from the forearm surface and finger joint angle signals during the key press action;

[0009] Step 2: Perform cascaded data preprocessing on the electromyography (EMG) signals on the forearm surface and the finger joint angle signals to obtain time-aligned time-frequency feature maps of the EMG signals and the finger joint angle signals, respectively.

[0010] Step 3: Use the time-frequency feature map of the finger joint angle signal from Step 2 as input to the teacher model for pre-training to obtain a trained joint model;

[0011] Step 4: Based on the time-frequency feature map of the electromyography signal obtained in Step 2, construct the student model; at the same time, use the joint model obtained in Step 3 as the teacher model to perform cross-modal knowledge distillation training, so as to realize the knowledge transfer from the teacher model to the student model.

[0012] Step 5: After completing cross-modal knowledge distillation, identify key gestures based on forearm surface electromyography signals to complete text input on the virtual numeric keypad.

[0013] Furthermore, in step 1, the key press actions adopt the blind typing rules based on the numeric keypad; the key press categories collected include 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, Num, division sign, asterisk, minus sign, plus sign, enter, and dot.

[0014] Furthermore, in step 2, the cascaded data preprocessing includes:

[0015] Signal denoising involves filtering, rectifying, and smoothing the surface electromyography (EMG) signal; and bandpass filtering, median filtering, and smoothing are applied to the finger joint angle signal.

[0016] Sampling rate synchronization: The surface electromyography signal is downsampled to a sampling frequency consistent with the finger joint angle signal;

[0017] Action detection and segment extraction: surface electromyography signal action segments are extracted based on sliding window root mean square threshold, and finger joint angle signal action segments are extracted based on angular velocity threshold.

[0018] Action segment alignment: Using the finger joint angle signal action segment as the anchor point, align the dual-modal action segments;

[0019] Sliding window segmentation uses overlapping sliding windows to segment aligned action segments;

[0020] Time-frequency feature extraction: Extract the time-domain and frequency-domain features of each window to construct a multi-channel pseudo-RGB feature map.

[0021] Furthermore, the time-domain features are the mean absolute value and root mean square, and the frequency-domain features are the median frequency, expressed as:

[0022] ;

[0023] ;

[0024] ;

[0025] In the formula, N is the number of sampling points of the signal; i is the i-th sampling point; It is the amplitude of the electromyography signal at the i-th sampling point; This represents the power spectral density value corresponding to the j-th frequency point;

[0026] The pseudo-RGB feature map has dimensions of C×W×H, where C=3 corresponds to three feature types, W is the number of sliding windows, and H is the number of signal channels.

[0027] Furthermore, both the teacher model in step 3 and the student model in step 4 adopt the Swin-Transformer network architecture.

[0028] Furthermore, the cross-modal knowledge distillation training process in step 4 includes:

[0029] Pre-training phase: Joint feature fragments and their corresponding ground truth labels are used as inputs to pre-train the joint model;

[0030] Training phase: Based on the joint model in the pre-training phase, the parameters of the model are frozen, and the electromyography model, which is the student, is constrained through multiple loss functions to achieve the process of cross-modal knowledge distillation.

[0031] Several loss functions include Kulbeck-Leibler divergence loss, interlayer distillation loss, and cross-entropy loss.

[0032] Furthermore, the Kolb-Leibler divergence loss is used to measure the difference in output distribution between the student model based on surface electromyography signals and the teacher model based on joint signals. By minimizing the difference between their class probability distributions, the student model can gradually approximate the output distribution of the teacher model, achieving effective knowledge transfer. The Kolb-Leibler divergence loss is expressed as:

[0033] ;

[0034] In the formula, L KLD Indicates KLD loss; z t and z s represents the probability distribution of the outputs of the teacher model and the student model, respectively; T is the temperature coefficient, higher temperatures effectively smooth the class probability distribution, helping the student model learn the implicit knowledge contained in the probability relationships between classes in the teacher model; C is the total number of classes; Softmax i (·) represents the Softmax probability corresponding to the i-th class.

[0035] Furthermore, interlayer distillation loss is introduced to enhance the student model's ability to learn intermediate feature representations from the teacher model. This interlayer distillation loss constrains the student model's feature representations across multiple intermediate layers in the Swin-T network, ensuring consistency with the corresponding feature outputs of the teacher model. This achieves structured knowledge transfer of intermediate features within the hierarchical semantic space. The interlayer distillation loss is expressed as:

[0036] ;

[0037] In the formula, and These represent the feature outputs of the student model and the teacher model at the k-th distillation layer, respectively; K is the total number of layers used for feature distillation.

[0038] All feature maps were normalized using the L2 norm to eliminate the influence of amplitude differences between the two modes.

[0039] Furthermore, cross-entropy loss is used to measure the difference between the student model's predictions and the corresponding true labels. Cross-entropy loss is expressed as:

[0040] ;

[0041] In the formula, Let represent the probability distribution of the student model's output for the i-th sample; The unique-hot encoded true label for category C; N is the total number of samples;

[0042] Cross-entropy loss ensures that the student model's output always accurately fits the true label during the distillation process.

[0043] The second aspect is a gesture recognition device that uses a virtual keyboard key gesture recognition method based on electromyography signals. The gesture recognition device includes a flexible strip, in which multiple channels of electromyography sensors are arranged along the length direction. An integrated module is also provided in the flexible strip. A power button is provided on the end face of the flexible strip, and a charging interface is provided on one side of the flexible strip. The flexible strip is connected end to end to form an arm ring structure.

[0044] The channel electromyography sensor includes a metal electrode array, a metal sub-electrode PCB board, and a housing. The metal sub-electrode PCB board is laid inside the housing, and the metal electrode array is provided on the metal sub-electrode PCB board. The metal electrode array includes multiple gold-plated spring needles and conductive metal caps arranged in an array. The gold-plated spring needles are located on the metal sub-electrode PCB board, and the conductive metal caps are located on the gold-plated spring needles and connected to the housing.

[0045] The technical solution adopted in this invention has the following beneficial effects:

[0046] In this invention, to improve the recognition accuracy of fine-grained key press actions, a teacher model is constructed by introducing joint motion signals. Knowledge transfer from the electromyography (EMG) model is performed through cross-modal knowledge distillation, thereby improving the recognition performance of EMG signals. Simultaneously, recognition is completed solely using EMG signals during the inference stage, effectively enhancing the system's wearability and real-time performance. Furthermore, a cascaded data preprocessing method is designed to extract time-frequency features and align actions from EMG and joint signals, providing high-quality input features for subsequent classification models. Based on the above methods, to adapt to different fine-grained tasks and muscles in different locations, this invention designs a dual-mode EMG armband for key press action recognition, thus achieving a high-precision, high-efficiency, and wearable virtual digital keyboard gesture recognition solution suitable for human-computer interaction scenarios without physical keyboards. Attached Figure Description

[0047] Figure 1 This is an overall flowchart of the real key gesture recognition method based on electromyography signals of the present invention;

[0048] Figure 2 A flowchart illustrating the structure of cascaded data preprocessing;

[0049] Figure 3 A schematic diagram of the structure for cross-modal knowledge distillation;

[0050] Figure 4 A comparison chart showing the accuracy of key press recognition before and after electromyographic signal distillation;

[0051] Figure 5 A schematic diagram of wearing an electromyography armband;

[0052] Figure 6 This is a frontal view of the electromyography (EMG) sensor.

[0053] Figure 7 A top-view cross-sectional view of the electromyography (EMG) sensor unit.

[0054] Figure 8 Flowchart of the electromyography sensor unit

[0055] Figure 9 This is a side-rear view of the electromyography arm loop.

[0056] 1. Channel electromyography sensor; 2. Integrated module; 3. Flexible band; 11. Metal electrode array; 111. Conductive metal cap; 112. Gold-plated spring pin; 12. Metal sub-electrode PCB board; 13. Housing; 21. Power button; 22. Charging interface. Detailed Implementation

[0057] To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0058] This invention provides a real key gesture recognition method and recognition device based on surface electromyography signals for input of virtual numeric keyboards, realizing an efficient, wearable, and high-precision virtual numeric keyboard gesture recognition method, which is suitable for scenarios without physical keyboard input.

[0059] like Figure 1 As shown, a virtual keyboard key gesture recognition method based on electromyography signals includes the following steps:

[0060] Step 1: Data Acquisition: During the execution of key presses, electromyographic signals of the forearm surface and finger joint angle signals are acquired simultaneously; the key presses are set according to the keyboard touch typing rules to ensure the standardization and consistency of the acquired key press action data, and there are a total of 17 key values.

[0061] Step 2: The joint signal and electromyography (EMG) signal modal signals are subjected to cascaded data preprocessing to achieve time alignment and time-frequency feature extraction of the two modalities, respectively obtaining time-aligned EMG signal time-frequency feature maps and finger joint angle signal time-frequency feature maps, providing high-quality input features for subsequent classification models.

[0062] Step 3: Using the time-frequency feature map of the finger joint angle signal obtained in Step 2 as input, construct the teacher model and pre-train it to obtain the trained joint model.

[0063] Step 4: Based on the time-frequency features of the electromyography signal obtained in Step 2, a student model is constructed; at the same time, the joint model obtained in Step 3 is used as the teacher model for cross-modal knowledge distillation training, realizing knowledge transfer from the teacher model to the student model, which effectively improves the recognition accuracy of the electromyography model.

[0064] Step 5: After completing cross-modal knowledge distillation, identify key gestures based on forearm surface electromyography signals to complete text input on the virtual numeric keypad.

[0065] In this embodiment, the key press actions in step 1 are set based on the touch typing rules of the numeric keypad; the collected key categories include: 50, 1, 2, 3, 4, 5, 6, 7, 8, 9, Num, division sign, asterisk, minus sign, plus sign, enter, and point, which are 17 key values.

[0066] like Figure 2 As shown, the cascaded data preprocessing in step 2 includes:

[0067] Signal denoising: Surface electromyography signals are filtered, rectified, and smoothed; finger joint angle signals are bandpass filtered, median filtered, and smoothed, effectively eliminating specific interference caused by factors such as power frequency noise and motion artifacts from the acquisition equipment;

[0068] Sampling Rate Synchronization: Since the sampling frequencies of joint EMG and surface EMG signals are 500 Hz and 1000 Hz, respectively, sampling rate synchronization is necessary to ensure accurate segment extraction, alignment, and feature analysis. To achieve this, a stepwise downsampling process is applied while preserving the essential components of the surface EMG signal. Following the Nyquist criterion, an anti-aliasing low-pass filter is used to preserve frequencies below 225 Hz, and then the EMG signal is downsampled to 500 Hz to match the joint signal rate.

[0069] Action detection and segment extraction: Action segments were extracted from surface electromyography signals based on the root mean square (RMS) threshold of the sliding time window, and action segments were extracted from finger joint angle signals based on the angular velocity (AV) threshold. The extraction results were filtered and constrained to ensure that the acquired electromyography and joint signals came from valid action segments, rather than resting states.

[0070] Action segment alignment: First, the start and end points of each segment's sEMG and joint signal are marked. To eliminate fixed time sets, initial alignment is performed based on the first segment start point of both modes. Considering that joint signals directly reflect key movements and that no missed detections occur during action detection, the nearest preceding 5-joint action segment is located using the start point of all joint action segments as the anchor point. If no corresponding sEMG action segment is detected in the previous stage, the mismatched joint segment is discarded to maintain consistency between the signal of interest and the sEMG action segment.

[0071] Sliding window segmentation: Overlapping sliding time windows are used to segment the time-aligned action segments. The length of the sliding window is set to 40 time points, and the step size is set to 20 time points. By segmenting action segments using sliding windows, the system can complete key action recognition based on local time segments without waiting for the complete action to finish, thereby reducing recognition latency and improving the system's real-time performance.

[0072] Time-frequency feature extraction: Extract the time-domain and frequency-domain features of each window to construct a multi-channel pseudo-RGB feature map. The time-frequency features include: time-domain features are Mean Absolute Value (MAV) and RMS, and frequency-domain features are Median Frequency (MF); the expressions are as follows:

[0073] ;

[0074] ;

[0075] ;

[0076] In the formula, N is the number of sampling points of the signal; i is the i-th sampling point; It is the amplitude of the electromyography signal at the i-th sampling point; This represents the power spectral density value corresponding to the j-th frequency point.

[0077] The pseudo-RGB feature map has dimensions of C×W×H, where C=3 corresponds to three feature types, W is the number of sliding windows, and H is the number of signal channels.

[0078] like Figure 3 As shown, both the teacher model in step 3 and the student model in step 4 adopt the Swin-Transformer (Swin-T) network architecture.

[0079] Step 4, cross-modal knowledge distillation training, consists of the following two processes:

[0080] (1) Pre-training stage: Joint feature fragments and their corresponding real labels are used as inputs for pre-training joint models.

[0081] (2) Training phase: Based on the joint model in the pre-training phase as the teacher model, its parameters are frozen, and the electromyography model as the student is constrained by multiple loss functions to realize the process of cross-modal knowledge distillation.

[0082] The loss function includes:

[0083] (1) Körbek-Leibler divergence loss: To enable the student model to better learn knowledge from the teacher model, this paper uses KLD loss to measure the difference in output distribution between the student model based on sEMG (surface electromyography) and the teacher model based on joint signals. By minimizing the difference between their class probability distributions, the student model can gradually approximate the output distribution of the teacher model, thereby achieving effective knowledge transfer. The corresponding formula is as follows:

[0084] ;

[0085] In the formula, L KLD Indicates KLD loss; z t and z srepresents the probability distribution of the outputs of the teacher model and the student model, respectively; T is the temperature coefficient (T=10 in this invention), higher temperatures can effectively smooth the category probability distribution, helping the student model learn the implicit knowledge contained in the probability relationships between categories in the teacher model; C is the total number of categories; Softmax i (·) represents the Softmax probability corresponding to the i-th class.

[0086] (2) Interlayer Distillation Loss: In addition to KLD loss, this paper introduces interlayer distillation loss (LD Loss) to further enhance the student model's ability to learn intermediate feature representations from the teacher model. This loss constrains the student model's feature representations in multiple intermediate layers (i.e., S_stage1, S_stage2, and S_stage3) of the Swin-T network to maintain consistency with the corresponding feature outputs of the teacher model (i.e., T_stage1, T_stage2, and T_5stage3), thereby achieving structured knowledge transfer of intermediate features in the hierarchical semantic space. The loss is defined as follows:

[0087] ;

[0088] In the formula, and represents the feature outputs of the student model and the teacher model at the k-th distillation layer (e.g., S_stage1-3 and T_stage1-3), respectively; K is the total number of layers used for feature distillation; all feature maps are normalized by their L2 norm to eliminate the influence of amplitude differences between the two modes; this loss prompts the student model to align its intermediate feature representations with the teacher model, thereby achieving structured knowledge transfer of intermediate layer features in the hierarchical semantic space.

[0089] (3) Cross-entropy loss: To ensure that the student model retains its ability to learn from the true labels during the distillation process, this paper introduces standard cross-entropy loss (CE Loss) to measure the difference between the student model's predictions and the corresponding true labels. The loss function is defined as follows:

[0090] ;

[0091] In the formula, Let represent the probability distribution of the student model's output for the i-th sample; represents the one-hot encoded true label for category C; N is the total number of samples. Cross-entropy loss ensures that the student model's output always accurately fits the true label during the distillation process, and is the most fundamental supervision signal in the entire loss function.

[0092] like Figure 3As shown, the three loss components are weighted with weights α=5, β=2, and γ=1 respectively, and together they constitute the composite loss function of the student model, which effectively promotes the transfer of knowledge from the joint model to the electromyography model, thereby realizing cross-modal knowledge distillation.

[0093] This invention significantly improves the accuracy of electromyography (EMG) models in recognizing similar key gestures through a cross-modal knowledge distillation strategy. Furthermore, since the method relies solely on the EMG model for recognition during the inference phase, it eliminates the need for vision, data gloves, or physical keyboards during the inference phase. Virtual key gesture recognition can be completed using only an EMG armband, enhancing the system's wearability, real-time performance, and environmental adaptability. It can be applied to virtual / augmented reality text input, intelligent prosthetic control, rehabilitation training interaction, contactless input on mobile terminals, and scenarios in industry, medical care, and rescue where traditional keyboards are inconvenient to use.

[0094] To verify the effectiveness of the cross-modal knowledge distillation method proposed in this invention and to evaluate the superiority of using Swin-T as the classification model, this paper conducts comparative experiments with typical classification models such as CNN, GengNet, and Transformer (ViT) to verify the performance advantage of Swin-T in electromyography (EMG) signal classification tasks. Simultaneously, a comparative analysis of the recognition results of the EMG model before and after distillation is performed.

[0095] like Figure 4 As shown, experimental results demonstrate that cross-modal knowledge distillation significantly improves performance across all models. Specifically, the average recognition accuracy after distillation for Swin-T reaches 78.05%, an improvement of 18.65% compared to before distillation, validating the effectiveness of the proposed method. Furthermore, regardless of whether distillation is performed before or after distillation, Swin-T's classification accuracy outperforms CNN, GengNet, and ViT, further demonstrating Swin-T's performance advantage in classification tasks.

[0096] A gesture recognition device using the above method includes a flexible band, in which multiple channels of electromyography sensors are arranged along the length direction, and an integrated module is also provided in the flexible band. A power button is provided on the end face of the flexible band, and a charging interface is provided on one side of the flexible band. The flexible band is connected end to end to form an arm ring structure.

[0097] The channel electromyography sensor includes a metal electrode array, a metal sub-electrode PCB board, and a housing. The metal sub-electrode PCB board is laid inside the housing, and the metal electrode array is located on the metal sub-electrode PCB board. The metal electrode array includes multiple gold-plated spring pins and conductive metal caps arranged in an array. The gold-plated spring pins are located on the metal sub-electrode PCB board, and the conductive metal caps are placed on the gold-plated spring pins and connected to the housing.

[0098] During the inference phase, key press actions are recognized solely through the electromyography armband.

[0099] refer to Figure 5 The button action recognition device includes a channel electromyography sensor 1 and an integrated module 2; multiple electromyography sensors 1 are connected to an integrated module 2 and form a ring structure that can be worn on the arm within a flexible band.

[0100] refer to Figure 6 The integral channel electromyography (EMG) sensor 1 within the flexible band 3 forms an EMG sensor unit with 16 channels. These 16 channels can be divided into odd-numbered and even-numbered channels, and the surface of the flexible band 3 has 74 metal sub-electrodes (corresponding to...). Figure 6 The metal electrode array 11 (composed of 1101-1172) can uniformly cover the forearm.

[0101] The aforementioned channel electromyography sensor 1 includes four metal sub-electrodes PCB board 12 and a housing 13.

[0102] refer to Figure 7 The metal electrode array 11 is disposed inside the housing 13 and includes a conductive metal cap 111 made of gold-plated copper and a gold-plated spring pin 112. The top of the gold-plated spring pin is in contact with the conductive metal cap, and the bottom is interference-fitted and vertically embedded in the through hole of the metal sub-electrode PCB board 12. Each channel electromyography sensor 1 is interconnected by wiring.

[0103] refer to Figure 8 The metal sub-electrode PCB board 12 is used to amplify and filter the electrical signal output by the gold-plated spring pin 112;

[0104] The outer shell 13 is made of PLA material by 3D printing and is used to fix and cover the metal electrode array 11 and the metal sub-electrode PCB board 13.

[0105] refer to Figure 9 The integrated module 2 integrates a battery unit, a main control unit, and a wireless communication unit, and the units integrated on the integrated module 2 are all commonly used components in the prior art.

[0106] Battery unit: The battery unit is used to power the main control unit, electromyography sensor module unit and wireless communication unit. The housing 13 of the integrated module is provided with a charging interface 22.

[0107] The main control unit is used for acquiring 16-channel electromyography signals; a power button 21 is provided on the outer shell 13 of the integrated module, and the power button 21 is used to control the power on and off of the main control unit by long-pressing.

[0108] The wireless communication unit uses Bluetooth to transmit the electromyographic signals processed by the main control unit to an external device.

[0109] Traditional uniform acquisition methods are prone to information redundancy and reduced resource utilization efficiency in tasks involving large muscle groups or significantly different movement sizes, while in recognizing small muscle groups or fine-grained movements, the signal discrimination ability is affected by unreasonable channel utilization. Therefore, this recognition device is equipped with two acquisition modes: alternating acquisition mode and synchronous acquisition mode. The alternating acquisition mode alternates between acquiring signals from odd-numbered and even-numbered channels, and is selected by briefly pressing the power button 21. The synchronous acquisition mode simultaneously acquires electromyographic signals from all 16 channels, and is selected by double-clicking the power button 21.

[0110] In tasks involving the recognition of large muscle groups or significant differences in movement, the electromyographic signals of adjacent channels exhibit high correlation. If 16 channels are always acquired simultaneously, channel information redundancy can easily occur, increasing the costs of data acquisition, transmission, and processing. Therefore, an alternating acquisition mode can be used to reduce redundancy and improve resource utilization efficiency. For small muscle group and fine-grained tasks such as virtual keyboard action recognition, the spatial distribution differences and coordinated changes between different channels play a crucial role in action discrimination. Therefore, using a 16-channel simultaneous acquisition mode in step 1 of the key recognition method can completely preserve multi-channel electromyographic features and inter-channel related information, thereby improving the feature representation ability and recognition accuracy of key movements.

[0111] The acquisition method can adaptively select the appropriate acquisition mode according to different task requirements and the electromyographic signal characteristics of different acquisition sites.

Claims

1. A method for recognizing virtual keyboard key gestures based on electromyographic signals, characterized in that, Key gesture recognition methods include: Step 1: Synchronously collect electromyographic signals from the forearm surface and finger joint angle signals during the key press action; Step 2: Perform cascaded data preprocessing on the electromyography (EMG) signals on the forearm surface and the finger joint angle signals to obtain time-aligned time-frequency feature maps of the EMG signals and the finger joint angle signals, respectively. Step 3: Use the time-frequency feature map of the finger joint angle signal from Step 2 as input to the teacher model for pre-training to obtain a trained joint model; Step 4: Based on the time-frequency feature map of the electromyography signal obtained in Step 2, construct the student model; at the same time, use the joint model obtained in Step 3 as the teacher model to perform cross-modal knowledge distillation training, so as to realize the knowledge transfer from the teacher model to the student model. Step 5: After completing cross-modal knowledge distillation, identify key gestures based on forearm surface electromyography signals to complete text input on the virtual numeric keypad.

2. The virtual keyboard key gesture recognition method based on electromyography signals according to claim 1, characterized in that, In step 1, the key presses are based on the touch typing rules of the numeric keypad; the key press categories collected include 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, Num, division sign, asterisk, minus sign, plus sign, enter, and dot.

3. The virtual keyboard key gesture recognition method based on electromyography signals according to claim 1, characterized in that, In step 2, the cascaded data preprocessing includes: Signal denoising involves filtering, rectifying, and smoothing the surface electromyography (EMG) signal; and bandpass filtering, median filtering, and smoothing the finger joint angle signal. Sampling rate synchronization: The surface electromyography signal is downsampled to a sampling frequency consistent with the finger joint angle signal; Action detection and segment extraction: surface electromyography signal action segments are extracted based on sliding window root mean square threshold, and finger joint angle signal action segments are extracted based on angular velocity threshold. Action segment alignment: Using the finger joint angle signal action segment as the anchor point, align the dual-modal action segments; Sliding window segmentation uses overlapping sliding windows to segment aligned action segments; Time-frequency feature extraction: Extract the time-domain and frequency-domain features of each window to construct a multi-channel pseudo-RGB feature map.

4. The virtual keyboard key gesture recognition method based on electromyography signals according to claim 3, characterized in that, The time-domain features are the mean absolute value and root mean square, and the frequency-domain features are the median frequency, expressed as: ； ； ； In the formula, N is the number of sampling points of the signal; i is the i-th sampling point; It is the amplitude of the electromyographic signal at the i-th sampling point; This represents the power spectral density value corresponding to the j-th frequency point; The pseudo-RGB feature map has dimensions of C×W×H, where C=3 corresponds to three feature types, W is the number of sliding windows, and H is the number of signal channels.

5. The virtual keyboard key gesture recognition method based on electromyography signals according to claim 1, characterized in that, Both the teacher model in step 3 and the student model in step 4 adopt the Swing-Transformer network architecture.

6. The virtual keyboard key gesture recognition method based on electromyography signals according to claim 1, characterized in that, The cross-modal knowledge distillation training process in step 4 includes: Pre-training phase: Joint feature fragments and their corresponding ground truth labels are used as inputs to pre-train the joint model; Training phase: Based on the joint model in the pre-training phase, the parameters of the model are frozen, and the electromyography model, which is used as the student, is constrained through multiple loss functions to realize the process of cross-modal knowledge distillation. Several loss functions include Kulbeck-Leibler divergence loss, interlayer distillation loss, and cross-entropy loss.

7. The virtual keyboard key gesture recognition method based on electromyography signals according to claim 6, characterized in that, The Kolb-Leibler divergence loss is used to measure the difference in output distribution between the student model based on surface electromyography (EMG) signals and the teacher model based on joint signals. By minimizing the difference between their class probability distributions, the student model can gradually approximate the output distribution of the teacher model, achieving effective knowledge transfer. The Kolb-Leibler divergence loss is expressed as: ； In the formula, L KLD Indicates KLD loss; z t and z s represents the probability distribution of the outputs of the teacher model and the student model, respectively; T is the temperature coefficient, higher temperatures effectively smooth the class probability distribution, helping the student model learn the implicit knowledge contained in the probability relationships between classes in the teacher model; C is the total number of classes; Softmax i (·) represents the Softmax probability corresponding to the i-th class.

8. The virtual keyboard key gesture recognition method based on electromyography signals according to claim 6, characterized in that, Interlayer distillation loss is introduced to enhance the student model's ability to learn intermediate feature representations from the teacher model. This loss constrains the student model's feature representations across multiple intermediate layers in the Swin-T network, ensuring consistency with the corresponding feature outputs of the teacher model. Structured knowledge transfer of intermediate features is achieved within the hierarchical semantic space. The interlayer distillation loss is expressed as: ； In the formula, and These represent the feature outputs of the student model and the teacher model at the k-th distillation layer, respectively; K is the total number of layers used for feature distillation. All feature maps were normalized using the L2 norm to eliminate the influence of amplitude differences between the two modes.

9. The virtual keyboard key gesture recognition method based on electromyography signals according to claim 6, characterized in that, Cross-entropy loss is used to measure the difference between the student model's predictions and the corresponding true labels. Cross-entropy loss is expressed as: ； In the formula, Let represent the probability distribution of the student model's output for the i-th sample; The unique hot-coded real label for category C; N is the total number of samples; Cross-entropy loss ensures that the student model's output always accurately fits the true label during the distillation process.

10. An electromyographic armband employing the virtual keyboard key gesture recognition method based on electromyographic signals as described in claim 1, characterized in that, The electromyography arm ring includes a flexible band (3), with multiple channels of electromyography sensors (1) arranged along the length direction inside the flexible band (3), and an integrated module (2) also provided inside the flexible band (3). A power button (21) is provided on the end face of the flexible band (3), and a charging interface (22) is provided on one side of the flexible band (3). Among them, the channel electromyography sensor (1) includes a metal electrode array (11), a metal sub-electrode PCB board (12) and a housing (13). The metal sub-electrode PCB board (12) is laid inside the housing (13), and the metal sub-electrode PCB board (12) is provided with a metal electrode array (11). The metal electrode array (11) includes multiple gold-plated spring needles (112) and conductive metal caps (111) arranged in an array. The gold-plated spring needles (112) are provided on the metal sub-electrode PCB board (12), and the conductive metal caps (111) are provided on the gold-plated spring needles (112) and connected to the housing.

Images