Wearable embedded gesture recognition system based on lightweight convolutional network model and STM32

Abstract

The invention discloses a wearable embedded gesture recognition system based on a lightweight convolutional network model and STM32, and belongs to the technical field of gesture recognition and embedded artificial intelligence. The system comprises an STM32, an upper computer, controlled equipment, a data acquisition module, a data preprocessing module, a lightweight convolutional network model training module, a model deployment module, an offline real-time reasoning module and an application module. The data acquisition module acquires a forearm surface electromyogram signal through a sensor and transmits the forearm surface electromyogram signal to the STM32, the STM32 transmits the signal to the upper computer, the data preprocessing module processes the signal to generate a sample with a label, the lightweight convolutional network model training module obtains a model through pre-training and lightweight optimization, the model deployment module burns the model to the STM32, and the model deployment module carries out model deployment. The off-line real-time reasoning module conducts reasoning through the STM32 and transmits a result to the application module, and the application module controls the controlled equipment to act. The lightweight convolutional network model is small in size, can be adapted to STM32 to realize embedded deployment, and can realize high-accuracy offline real-time gesture recognition under low power consumption.

Description

Technical Field

[0001] This invention belongs to the field of intersectional technology of gesture recognition and embedded artificial intelligence, specifically involving a wearable embedded gesture recognition technology based on surface electromyography signals, a lightweight convolutional network model and an STM32 microcontroller. Background Technology

[0002] Gesture recognition technology has become a research focus in recent years. Its applications are wide-ranging, covering areas such as human-computer interaction, prosthetic and robotic arm control, and sign language communication. It also has significant application value in fields such as medicine, industry, and the military, providing important technical support for improving human-computer interaction efficiency and expanding equipment control methods.

[0003] Currently, mainstream gesture recognition technologies can be divided into three categories. The first category is computer vision-based methods, which collect gesture information from images or videos and then recognize it. However, in practical applications, these methods are greatly affected by the environment; changes in lighting, differences in shooting angles, and external obstructions can all interfere with the recognition process, leading to decreased accuracy and limiting their application in complex environments. The second category is data glove-based methods, which collect gesture-related data by installing various sensors on gloves to achieve recognition. However, data gloves are expensive, and related technology research progress is slow, making it difficult to meet the needs of large-scale application. The third category is electromyography (EMG) signal-based methods. Among these, surface EMG signals are considered to have broad application prospects due to their simple acquisition method—requiring only the placement of electrode pads on the skin surface—low cost, and easy data processing.

[0004] Gesture recognition methods based on surface electromyography (EMG) signals can be further divided into traditional machine learning methods and deep learning methods. Traditional machine learning methods require preprocessing and manual feature extraction of the acquired EMG signals before classification algorithms are used for recognition. These methods are relatively stable in small sample scenarios, but they are highly dependent on feature engineering. When faced with complex gestures, it is difficult to extract comprehensive and effective features, resulting in insufficient recognition accuracy. Deep learning methods adopt an end-to-end processing model, which can automatically learn data features without manual extraction. However, these methods have high requirements for data volume and computing resources. Existing commonly used deep learning algorithms also suffer from poor parallelism, a large number of parameters, high optimization difficulty, and large model size, which are not conducive to application on resource-constrained devices.

[0005] Furthermore, most current gesture recognition research is conducted on computers. While computers provide ample computing resources, they cannot meet the demands of wearable devices for low power consumption, high portability, and real-time response. How to effectively deploy gesture recognition models on resource-constrained microcontrollers to achieve high-accuracy, low-latency offline real-time gesture recognition in embedded environments, while also meeting the needs of wearable devices, remains a critical technological bottleneck that urgently needs to be overcome in this field, hindering the widespread application of gesture recognition technology in more practical scenarios. Summary of the Invention

[0006] The purpose of this invention is to provide a wearable embedded gesture recognition system based on a lightweight convolutional network model and STM32, which solves the problems of existing deep learning gesture recognition models being large in size, requiring high computing resources, and difficult to deploy on microcontrollers, as well as the fact that most gesture recognition is performed on computers and cannot meet the low power consumption, high portability, and real-time response requirements of wearable devices.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A wearable embedded gesture recognition system based on a lightweight convolutional network model and an STM32 microcontroller includes: an STM32 microcontroller, a host computer, a controlled device, and a data acquisition module, a data preprocessing module, a lightweight convolutional network model training module, a model deployment module, an offline real-time inference module, and an application module. The data acquisition module is signal-connected to the STM32 microcontroller to transmit surface electromyography (EMG) signals. The STM32 interacts with the host computer via a serial port to transmit signal data. The host computer is signal-connected to the data preprocessing module, the lightweight convolutional network model training module, and the model deployment module to transmit data and models. The model deployment module is signal-connected to the STM32 microcontroller to program the model. The STM32 is signal-connected to the offline real-time inference module to provide the model and signals. The offline real-time inference module interacts with the application module to transmit inference results. The application module is signal-connected to the controlled device to transmit control signals.

[0009] In one possible implementation, the data acquisition module includes multiple electromyography (EMG) signal sensors and an elastic band. The data acquisition module uses the elastic band to fix the multiple EMG signal sensors at equal intervals to the middle of the user's forearm. The multiple EMG signal sensors collect surface EMG signals when the user performs a specified gesture. The collected surface EMG signals are transmitted to the STM32 to obtain raw surface EMG signal data. Part of the raw surface EMG signal data is stored in the host computer.

[0010] In one possible implementation, the data preprocessing module performs a truncation operation on the raw surface electromyography (EMG) signal data stored in the host computer for each round to remove initial transient noise and obtain the truncated active segment data.

[0011] In one possible implementation, the data preprocessing module uses the truncated active segment data to first filter it with a high-pass filter to eliminate baseline drift, obtaining high-pass filtered data; then it uses a power frequency notch filter to filter the high-pass filtered data to eliminate power frequency interference, obtaining filtered data.

[0012] In one possible implementation, the data preprocessing module uses a sliding window to segment the filtered data and associates it with corresponding gesture action tags to generate data samples with gesture action tags, which are then stored in the host computer.

[0013] In one possible implementation, the lightweight convolutional network model training module uses the data samples with gesture action labels stored on the host computer to select a portion of the data samples as the training group and the remaining data samples as the test group; then, the training group data is divided into a training set and a validation set using a stratified sampling strategy to ensure that the distribution of samples of each category is balanced, thus obtaining the divided training data.

[0014] In one possible implementation, the lightweight convolutional network model training module pre-trains based on the partitioned training data to construct a network containing multiple residual blocks. Each residual block is equipped with a dilated causal convolutional layer and uses a convolutional kernel of a set size. The gradient vanishing problem is alleviated through residual connections. The network parameters are optimized using activation functions and optimizers combined with dynamic learning rate adjustment and gradient clipping strategies to obtain an initial gesture recognition model.

[0015] In one possible implementation, the lightweight convolutional network model training module evaluates the importance of each convolutional layer channel in the initial gesture recognition model based on the L1 norm, removes channels with lower weights according to a set ratio, reconstructs a compact model, gradually restores model performance through progressive fine-tuning, obtains a lightweight gesture recognition model, and transmits the lightweight gesture recognition model to the model deployment module.

[0016] In one possible implementation, the model deployment module compresses the lightweight gesture recognition model using a dedicated plugin and development tools to reduce its size, then converts the compressed model to a format compatible with the STM32 to obtain a compatible model file; the compatible model file is then burned into the storage unit of the STM32 to achieve the deployment of the lightweight gesture recognition model on the STM32.

[0017] In one possible implementation, the offline real-time inference module acquires surface electromyography (EMG) signals from the middle of the user's forearm in real time using the STM32 to obtain real-time raw EMG signals. The real-time raw EMG signals are then processed according to the filtering method of the data preprocessing module to obtain real-time filtered data. This filtered data is then input to a lightweight gesture recognition model deployed on the STM32, which performs offline inference calculations to generate real-time gesture inference results, which are then transmitted to the application module. The application module parses the controlled operation instructions based on the real-time gesture inference results, generates corresponding control signals, and transmits them to the controlled device, causing the controlled device to execute the corresponding action.

[0018] Compared with existing technologies, the advantages of this invention are as follows: The lightweight convolutional network model in this invention constructs a network with multiple residual blocks during the pre-training stage. Each residual block is equipped with two dilated causal convolutional layers and uses convolutional kernels of a specified size. The gradient vanishing problem is alleviated by residual connections, and long-range dependencies are captured by increasing the dilation factor, enabling the model to fully learn the effective features of surface electromyography signals. In the subsequent lightweight optimization stage, the importance of convolutional layer channels is evaluated based on the L1 norm, low-weight channels are removed, and performance is gradually fine-tuned to recover the model, which significantly reduces the model size and the number of parameters, allowing the model to adapt to resource-constrained microcontrollers and solving the problems of large size and high computational resource requirements of traditional deep learning models.

[0019] The model deployment module uses dedicated plugins and development tools to compress and convert the lightweight model before burning it to the STM32 microcontroller. Combined with the low-power hardware characteristics of the STM32, it achieves efficient deployment of the model on the embedded side. The offline real-time inference module collects surface electromyography signals in real time through the STM32, processes the signals using the filtering method of the data preprocessing module, and then inputs them into the local model for inference. It does not rely on external computing devices, which reduces the overall power consumption of the system and reduces inference latency, meeting the requirements of wearable devices for low power consumption, high portability, and real-time response.

[0020] The data acquisition module uses elastic straps to fix multiple electromyography (EMG) sensors at equal intervals in the middle of the forearm, ensuring the stability and consistency of signal acquisition. The data preprocessing module removes initial transient noise, eliminates baseline drift through high-pass filtering, and eliminates power frequency interference through power frequency notch filtering. Then, it generates labeled samples through sliding window segmentation. This multi-step processing improves the data quality of surface EMG signals, providing high-quality data support for subsequent model training. Combined with the feature learning capabilities of the lightweight convolutional network model, it significantly improves the accuracy of gesture recognition and ensures the reliability of various gesture recognition results.

[0021] The application module can parse the corresponding controlled operation instructions based on the gesture inference results output by the offline real-time inference module and generate control signals to transmit to the controlled device. This realizes the direct mapping of gesture recognition results to device control, enabling devices such as prosthetic hands and drones to be flexibly controlled through gestures. This expands the practical application scope of gesture recognition technology in human-computer interaction scenarios and promotes the development of this technology towards practical application. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a structural diagram of a wearable embedded gesture recognition system based on a lightweight convolutional network model and STM32 according to an embodiment of the present invention.

[0024] Figure 2 This is a schematic diagram illustrating the placement of the electromyography (EMG) sensor during data collection by the subject in an embodiment of the present invention.

[0025] Figure 3 This is a schematic diagram of seven hand gestures that require data collection in an embodiment of the present invention;

[0026] Figure 4 This is a schematic diagram of the sliding window segmentation method in an embodiment of the present invention;

[0027] Figure 5 This is a framework diagram of a lightweight convolutional network model in an embodiment of the present invention. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0029] Example:

[0030] It should be noted that the terms "comprising" and "having" and any variations thereof in the embodiments of the present invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such processes, methods, products, or devices.

[0031] This invention provides a wearable embedded gesture recognition system based on a lightweight convolutional network model and an STM32 microcontroller. The system includes an STM32 microcontroller, a host computer, a controlled device, and several modules: a data acquisition module, a data preprocessing module, a lightweight convolutional network model training module, a model deployment module, an offline real-time inference module, and an application module. The data acquisition module is connected to the STM32 microcontroller to transmit surface electromyography (EMG) signals. The STM32 interacts with the host computer via a serial port to transmit signal data. The host computer is connected to the data preprocessing module, the lightweight convolutional network model training module, and the model deployment module to transmit data and models. The model deployment module is connected to the STM32 microcontroller to program the model. The STM32 is connected to the offline real-time inference module to provide the model and signals. The offline real-time inference module interacts with the application module to transmit inference results. The application module is connected to the controlled device to transmit control signals.

[0032] Specifically, the STM32 is the STM32F429 microcontroller, the host computer is a computer capable of data processing and model training, and the controlled device is a device that can receive control signals and execute actions. The data acquisition module can be an electromyography (EMG) sensor, which connects to the STM32's signal input interface via wires. The STM32 connects to the host computer via a USART serial port (via a USB-to-serial converter chip). The data preprocessing module and the lightweight convolutional network model training module are located inside the host computer. The model deployment module connects to the STM32 via a USART / USB interface. The offline real-time inference module connects to the application module via an SPI bus, and the application module connects to the controlled device via a wireless communication module. After the system starts, the data acquisition module collects surface electromyography (EMG) signals and transmits them to the STM32. The STM32 then transmits the signal data to the host computer. The host computer processes the data through the data preprocessing module, the lightweight convolutional network model training module trains the model, and the model deployment module burns the model into the STM32. The STM32 provides the model and real-time filtered EMG signals to the offline real-time inference module. After completing the inference, the offline real-time inference module transmits the results to the application module. The application module sends control signals to the controlled device, realizing a complete gesture recognition and device control process.

[0033] In one embodiment, the data acquisition module includes multiple electromyography (EMG) signal sensors and an elastic band. The data acquisition module uses the elastic band to fix the multiple EMG signal sensors at equal intervals to the middle of the user's forearm. The multiple EMG signal sensors collect surface EMG signals when the user performs a specified gesture. The collected surface EMG signals are transmitted to the STM32 to obtain raw surface EMG signal data. Part of the raw surface EMG signal data is stored in the host computer.

[0034] Specifically, the electromyography (EMG) signal sensor is a surface EMG sensor, and the elastic band is a breathable fabric band. The raw surface EMG signal data is unprocessed, directly acquired EMG signal data. There are 8 EMG signal sensors, with a spacing of 2 cm between adjacent sensors, and the elastic band is 3 cm wide. Before use, the user's mid-forearm skin is cleaned to remove oil and impurities. The 8 EMG signal sensors are arranged at equal intervals and secured with the elastic band to ensure that the sensors are in close contact with the skin. The user performs 7 specified hand gestures in sequence as instructed. The sensors collect signals at a sampling rate of 1000 Hz and transmit them to the STM32 in real time. The STM32 then transmits the raw surface EMG signal data to the host computer for storage via serial port.

[0035] In one embodiment, the data preprocessing module performs a truncation operation on the raw surface electromyography (EMG) signal data stored in the host computer for each round to remove initial transient noise, thereby obtaining the truncated active segment data.

[0036] Specifically, the initial transient noise is the signal noise corresponding to non-target actions caused by physiological response delays when the user switches gestures. The raw surface electromyography (EMG) signal data for each round is the signal data within a single acquisition cycle, and the truncated data is the effective signal data after removing the initial noise. The data preprocessing module reads the raw EMG signal data from the host computer, segments it according to the acquisition duration of each round, automatically calculates and truncates the first 1 / 6 of the data segment for each round, retains the remaining 5 / 6 of the data as the truncated data, and stores it in a designated area for subsequent processing.

[0037] In one embodiment, the data preprocessing module uses the truncated active segment data to first filter it with a high-pass filter to eliminate baseline drift, obtaining high-pass filtered data; then it uses a power frequency notch filter to filter the high-pass filtered data to eliminate power frequency interference, obtaining filtered data.

[0038] Specifically, baseline drift is the low-frequency drift component in the electromyography signal caused by unstable electrode contact and slow physiological changes. Power frequency interference is electromagnetic interference generated by surrounding 50 Hz power lines. The high-pass filtered data is the signal data after removing baseline drift, and the filtered data is the final preprocessed data after removing both types of interference. The data preprocessing module calls a 4th-order Butterworth high-pass filter to input the truncated data, filters it according to the set cutoff frequency, and outputs the high-pass filtered data. Then, it inputs the high-pass filtered data into a power frequency notch filter to suppress power frequency interference according to preset parameters, outputs the filtered data, and stores it.

[0039] In one embodiment, the data preprocessing module segments the filtered data using a sliding window and associates it with corresponding gesture action tags to generate data samples with gesture action tags, which are stored in the host computer.

[0040] Specifically, the sliding window is a signal segmentation window with a fixed length and sliding step size. Gesture action labels are information identifying the corresponding gesture actions of the data samples. Data samples with gesture action labels are datasets containing signal segments and corresponding gesture identifiers. The sliding window length is 200 sampling points, and the sliding step size is 50 sampling points. The gesture action labels are numbers 1 to 7, each corresponding to one of the seven specified gesture actions. The data preprocessing module reads the filtered data, segments it segment by segment from the starting position according to the set window length and step size, automatically associating each segment with the corresponding gesture action label to form data samples. This process is repeated until all data processing is complete, generating batch samples which are stored on the host computer for training.

[0041] In one embodiment, the lightweight convolutional network model training module uses data samples with gesture action labels received by the host computer to select a portion of the data samples as the training group and the remaining data samples as the test group; then, the training group data is divided into a training set and a validation set using a stratified sampling strategy to ensure that the distribution of samples of each category is balanced, thus obtaining the divided training data.

[0042] Specifically, the training group is the sample set used for model training, and the test group is the sample set used for model performance testing. The stratified sampling strategy involves drawing a specified proportion of samples according to the proportion of samples in each category. The resulting training data consists of a dataset containing both training and validation sets. For each subject, three sets of data are collected, two sets are selected and merged to form the training group, and the remaining set is used as the test group. Stratified sampling is applied to the training group data, extracting 70% of the samples corresponding to each gesture label as the training set and 30% as the validation set. After partitioning, the data is integrated into the final training data for subsequent model pre-training.

[0043] In one embodiment, the lightweight convolutional network model training module pre-trains based on the partitioned training data to construct a network containing multiple residual blocks. Each residual block is equipped with a dilated causal convolutional layer and uses a convolutional kernel of a set size. The gradient vanishing problem is alleviated through residual connections. The network parameters are optimized by using activation functions and optimizers combined with dynamic learning rate adjustment and gradient clipping strategies to obtain an initial gesture recognition model.

[0044] Specifically, a residual block is a network structural unit containing residual connections; a dilated causal convolutional layer is a layer that expands the receptive field through dilation factors and follows causal relationships during convolution; a convolutional kernel is a matrix that extracts data features; a residual connection is a connection method that directly adds the input to the output after nonlinear transformation; an activation function is a function that introduces nonlinear transformation capabilities; an optimizer is an algorithm that adjusts network parameters to minimize the loss function; and the initial gesture recognition model is a pre-trained model with preliminary classification capabilities. The network contains 6 residual blocks, each with two dilated causal convolutional layers; the kernel size is 3; the activation function is the ReLU function; the optimizer is the Adam optimizer; and the dilation factors increase in powers of 2, successively being 1, 2, 4, 8, 16, and 32. The lightweight convolutional network model training module loads the partitioned training data, constructs a network architecture with 6 residual blocks, and configures convolutional layers, convolutional kernels, dilation factors, residual connections, activation functions, and optimizers. The training set data is input into the network for training, and the effect is monitored in real time using the validation set. The parameters are adjusted according to the loss value. After training, the initial gesture recognition model is obtained.

[0045] In one embodiment, the lightweight convolutional network model training module evaluates the importance of each convolutional layer channel in the initial gesture recognition model based on the L1 norm, removes channels with lower weights according to a set ratio, reconstructs a compact model, gradually restores model performance through progressive fine-tuning, obtains a lightweight gesture recognition model, and transmits the lightweight gesture recognition model to the model deployment module.

[0046] Specifically, the L1 norm is the sum of the absolute values ​​of all elements in a vector, used to measure the importance of convolutional layer channel weights. The set ratio is the proportion of low-weight channels removed. Progressive fine-tuning is a training method that gradually adjusts parameters to restore model performance. The lightweight gesture recognition model is a small-sized, low-computation model after channel pruning and parameter optimization. The set ratio is 50%; the initial learning rate for progressive fine-tuning is 0.0001, gradually decreasing with each training epoch. The lightweight convolutional network model training module calls the L1 norm calculation function to evaluate the channel weights of each convolutional layer in the initial model, sorting and filtering low-weight channels; removing 50% of them, reconstructing the network based on the remaining channels to form a temporary model; progressive fine-tuning of the temporary model using training and validation set data, monitoring performance metrics; stopping when performance recovers to near the level of the initial model, resulting in the lightweight gesture recognition model; and transmitting it to the model deployment module via a data transmission interface.

[0047] In one embodiment, the model deployment module compresses the lightweight gesture recognition model using a dedicated plugin and development tools to reduce its size, then converts the compressed model to a format suitable for the STM32, resulting in an adapted model file. The adapted model file is then burned into the storage unit of the STM32, thereby deploying the lightweight gesture recognition model on the STM32.

[0048] Specifically, the dedicated plugin is a software plugin specifically for STM32 model deployment; the development tool is professional software for model processing and programming; the storage unit is the STM32's internal flash memory for storing model files and data; and the adapted model file is a compressed and format-converted file that meets the STM32's operational requirements. The dedicated plugin is the X-CUBE-AI plugin in the STM32CubeMX software; the development tool is the STM32 Keil5 software; and the STM32's storage unit is a 2MB flash memory. The model deployment module receives the lightweight model, launches the STM32CubeMX software, and calls the X-CUBE-AI plugin. It compresses the model by removing redundant parameters and quantization weights, converting it to a binary format supported by STM32, resulting in the adapted model file. The dedicated plugin sends a programming command to write the adapted model file to the STM32's flash memory. After programming, a verification process is performed to confirm that the file is stored intact and can be read normally, thus completing the model deployment.

[0049] In one embodiment, the offline real-time inference module acquires surface electromyography (EMG) signals from the middle of the user's forearm in real time using the STM32 to obtain real-time raw EMG signals. The real-time raw EMG signals are processed according to the filtering method of the data preprocessing module to obtain real-time filtered data. The real-time filtered data is input to a lightweight gesture recognition model deployed on the STM32, and the model is invoked to perform offline inference calculations to generate real-time gesture inference results, which are then transmitted to the application module. The application module parses the controlled operation instructions based on the real-time gesture inference results, generates corresponding control signals, and transmits them to the controlled device, causing the controlled device to perform the corresponding actions.

[0050] Specifically, offline real-time inference refers to the process of completing inference in real time on local hardware without relying on external networks and devices. The controlled operation command is the device control command corresponding to the gesture action, and the control signal is the electrical or wireless signal sent by the application module to the controlled device. The controlled device is a prosthetic hand or a drone; for example, when the inference result is gesture 1, the controlled operation command is for the drone to turn, and the control signal is a wireless signal of a specific frequency; when the inference result is gesture 3, the controlled operation command is for the prosthetic hand to clench its fist, and the control signal is a high-level signal. After the offline real-time inference module is started, it acquires the user's forearm electromyography (EMG) signals in real time through the STM32 signal acquisition interface to form real-time raw EMG signals. Following the filtering method of the data preprocessing module, it calls the STM32's internal filtering algorithm to perform high-pass filtering and power frequency notch filtering to obtain real-time filtered data. This data is then input into a lightweight model deployed on the STM32, which performs rapid offline inference and outputs real-time gesture inference results. These results are transmitted to the application module via the data bus. The application module parses the controlled operation commands according to the preset gesture-command mapping relationship and generates control signals according to the controlled device communication protocol. These signals are transmitted to the controlled device through the communication interface. Upon receiving the signals, the controlled device executes the corresponding actions, completing one gesture recognition and device control cycle.

[0051] The technical solutions of the embodiments of the present invention will be described below with reference to the accompanying drawings. Please refer to them. Figure 1 This invention proposes a wearable embedded gesture recognition system based on a lightweight convolutional network model and STM32. The system consists of three parts: an STM32, a host computer, and a controlled device, as well as a data acquisition module, a data preprocessing module, a lightweight convolutional network model training module, a model deployment module, an offline real-time inference module, and an application module.

[0052] The STM32 is primarily responsible for the following tasks: acquiring surface electromyography (EMG) signals; transmitting the signal data to the host computer via serial port; deploying the trained model on the host computer; filtering and denoising the acquired signals and performing offline real-time gesture inference; and sending the real-time gesture inference results to the controlled device.

[0053] The host computer is mainly responsible for the following tasks: receiving surface electromyography signal data from the STM32; training the data using a lightweight convolutional network to obtain a gesture recognition model; compressing and converting the model and deploying it to the STM32.

[0054] The controlled device is mainly responsible for receiving real-time gesture inference results from the STM32 and responding accordingly.

[0055] The data acquisition module is used to collect surface electromyographic signals of the skin in the middle of the forearm when the subject performs a specified hand gesture, and stores them in a certain format. In this embodiment, the specific working steps of the data acquisition module are as follows:

[0056] (1) Skin pretreatment: After connecting the electromyography signal sensor, STM32 microcontroller and other equipment, use alcohol swabs to wipe the skin surface of the middle of the subject's forearm to reduce skin impedance and improve data quality.

[0057] (2) such as Figure 2 As shown, N electromyography (EMG) sensors were placed at equal intervals in the middle of the subject's forearm (i.e., surface EMG signal data from N channels were collected). Figure 2 (N=8), and use elastic straps to fix it to prevent excessive displacement during data acquisition.

[0058] (3) During data collection, subjects should maintain a consistent upright sitting posture and a stable body position, placing their elbows on the table and tilting their forearms forward at an angle of approximately 45° to the table. Following the images and text instructions on the data collection UI interface, data should be collected sequentially as follows: Figure 3 The surface electromyography (EMG) signal data of the seven hand gestures shown are at a sampling rate of The frequency was 1000Hz. The collected data was transmitted to the host computer in real time and stored in a specific format. Each gesture was collected in 10 rounds, with each round lasting 3 seconds and a 5-second rest period between rounds. Each subject collected a total of 3 complete sets of data. To prevent muscle fatigue, subjects were required to rest for approximately 2 minutes after collecting data for each gesture and approximately 10 minutes after collecting each complete set of data.

[0059] The data preprocessing module performs three operations on the original signal data: truncation, filtering and noise reduction, and sliding window segmentation, and saves the processed data as a new data file. In this embodiment, the specific working steps of the data preprocessing module are as follows:

[0060] (1) In this embodiment, data is collected only when the subject performs the target gesture action, and collection stops during the rest phase. However, due to the unavoidable physiological response delay during human-computer interaction, the actual collected data may still contain noise from the transition phase of the gesture action. Specifically, the subject needs to switch between the target gesture action and the rest action according to the image instructions on the interface, that is, the subject switches the gesture action after the interface image is switched. If the subject is not familiar with the data collection process, there may be a certain delay in the action switching, resulting in inaccurate data collected at the beginning of the collection phase. Therefore, in order to improve data quality, the first 1 / 6 of the data in each round (i.e., the first 0.5 seconds of data in each round) is discarded, and only the last 5 / 6 (i.e., the last 2.5 seconds of data in each round) is retained as valid data.

[0061] (2) After data truncation, a fourth-order Butterworth high-pass filter is first used to filter each channel of the truncated signal data to eliminate baseline drift. Filter cutoff frequency =20Hz, the transfer function is:

[0062] Among them, coefficient Calculated using the butter function in Python;

[0063] Subsequently, a 50Hz power frequency notch filter was used to perform notch filtering on the high-pass filtered signal to eliminate power frequency interference. The filter quality factor was... =20, the transfer function is:

[0064] in, =50Hz, , , where is the sampling interval;

[0065] The high-pass filter and notch filter are implemented using zero-phase filtering, specifically:

[0066] For signals First, perform forward filtering to obtain And then The final output is obtained by performing inverse filtering. ,satisfy:

[0067] Here, filtfilt is a function in Python; the method can eliminate the phase distortion that may be introduced by traditional causal filtering, thereby preserving the integrity of the temporal characteristics of surface electromyography signals;

[0068] (3) After completing the filtering and denoising process, a sliding window is used to segment the truncated and filtered data to obtain a large number of data samples corresponding to different gesture action labels. The principle is as follows: Figure 4 As shown. This embodiment uses length =200 sampling points, sliding step size A sliding window of 50 sampling points is used for segmentation, satisfying:

[0069] in, For the first Signal data for each window, The filtered signal data, This represents the length of a single sample before segmentation.

[0070] The lightweight convolutional network model training module uses preprocessed data to train the gesture recognition model on the host computer and saves the trained model. In this embodiment, the specific working steps of the lightweight convolutional network model training module are as follows:

[0071] (1) For the three sets of data collected from each subject, two sets were selected as the training set and the other set was selected as the test set. After merging the data from the two training sets, a stratified sampling strategy was used to divide them into a training set (70%) and a validation set (30%) to ensure a balanced distribution of samples in each category.

[0072] (2) Use a lightweight convolutional network model to train the data, such as Figure 5 As shown, the training process comprises two stages: pre-training and lightweight optimization. In the pre-training stage, a network with six residual blocks is constructed. Each residual block has two dilated causal convolutional layers with kernels of size 3. The dilation factors increase exponentially by powers of 2 (1, 2, 4, 8, 16, 32), allowing the receptive field to cover longer input sequences to capture long-range dependencies and mitigating the vanishing gradient problem through residual connections. ReLU is used as the activation function, and the Adam optimizer, combined with dynamic learning rate adjustment and gradient clipping strategies, is employed to optimize the parameters. After pre-training, the lightweight optimization stage begins. The importance of each convolutional layer channel is evaluated based on the L1 norm, and channels with lower weights are removed by 50%. A more compact model is then reconstructed, and progressive fine-tuning gradually restores model performance, achieving a balance between model complexity and recognition accuracy. Through these two training stages, a lightweight, high-accuracy gesture classification model is finally obtained.

[0073] (3) After the model training is completed, a .pth format file containing the model structure and parameters is obtained. Convert it into a .onnx format model file and save it.

[0074] (4) The model was initially tested and evaluated using data from the test group. The evaluation system adopted multi-dimensional indicators, including precision, recall, and F1 score for each gesture category, as well as the overall classification accuracy of all gesture data. The table below shows the average evaluation indicators of the subjects' model for classifying and recognizing 7 gestures in this embodiment:

[0075] Furthermore, in this embodiment, the overall average accuracy of the model used by the subjects for classifying and recognizing seven gestures reached 98.4%. These evaluation results demonstrate that the lightweight convolutional network model designed in this invention can achieve good results in gesture recognition tasks based on upper limb surface electromyography signals.

[0076] The model deployment module deploys the trained model to the STM32 microcontroller, facilitating offline real-time gesture recognition on the microcontroller. In this embodiment, the module uses the X-CUBE-AI plugin in STM32CubeMX software and tools such as STM32 Keil5 to compress and convert the trained gesture recognition model, further lightweighting the model by reducing its size and memory usage. Subsequently, the model is burned into the STM32F429 microcontroller, completing the model deployment.

[0077] The offline real-time inference module uses an STM32 microcontroller to acquire surface electromyography (EMG) signal data in real time, performs filtering and noise reduction processing on the microcontroller, inputs the processed data into a locally deployed model for offline inference, and then sends the inference results to the controlled device. In this embodiment, the specific working steps of the offline real-time inference module are as follows:

[0078] (1) Use an STM32F429 microcontroller to collect electromyographic signal data on the surface of the skin in the middle of the user's forearm in real time and store it in the microcontroller;

[0079] (2) Implement filtering and noise reduction processing on the microcontroller;

[0080] (3) Input the processed data into the model deployed locally on the microcontroller for offline real-time gesture recognition, and send the inference results to the controlled device.

[0081] The application module receives real-time gesture inference results from the STM32 microcontroller and responds accordingly. For example, a prosthetic hand may make a fist or a peace sign based on the inference results, or a drone may make a turning or acceleration action based on the inference results.

[0082] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0083] The above embodiments are merely illustrative of the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made based on the essence of the content of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A wearable embedded gesture recognition system based on a lightweight convolutional network model and STM32, characterized in that, include: The system comprises an STM32 microcontroller, a host computer, a controlled device, and data acquisition, data preprocessing, lightweight convolutional network model training, model deployment, offline real-time inference, and application modules. The data acquisition module is connected to the STM32 microcontroller to transmit surface electromyography (EMG) signals. The STM32 interacts with the host computer via a serial port to transmit signal data. The host computer is connected to the data preprocessing, lightweight convolutional network model training, and model deployment modules to transmit data and models. The model deployment module is connected to the STM32 microcontroller to program the model. The STM32 is connected to the offline real-time inference module to provide the model and signals. The offline real-time inference module interacts with the application module to transmit inference results. The application module is connected to the controlled device to transmit control signals.

2. The wearable embedded gesture recognition system based on a lightweight convolutional network model and STM32 according to claim 1, characterized in that, The data acquisition module includes multiple electromyography (EMG) signal sensors and an elastic band. The data acquisition module uses the elastic band to fix the multiple EMG signal sensors at equal intervals to the middle of the user's forearm. The multiple EMG signal sensors collect surface EMG signals when the user performs a specified gesture. The collected surface EMG signals are transmitted to the STM32 to obtain raw surface EMG signal data. Some of the raw surface EMG signal data is stored in the host computer.

3. The wearable embedded gesture recognition system based on a lightweight convolutional network model and STM32 according to claim 1, characterized in that, The data preprocessing module performs a truncation operation on the raw surface electromyography (EMG) signal data stored in the host computer for each round to remove initial transient noise and obtain the truncated active segment data.

4. The wearable embedded gesture recognition system based on a lightweight convolutional network model and STM32 according to claim 3, characterized in that, The data preprocessing module uses the truncated active segment data to first filter it with a high-pass filter to eliminate baseline drift, obtaining high-pass filtered data; then it uses a power frequency notch filter to filter the high-pass filtered data to eliminate power frequency interference, obtaining filtered data.

5. The wearable embedded gesture recognition system based on a lightweight convolutional network model and STM32 according to claim 4, characterized in that, The data preprocessing module segments the filtered data using a sliding window and associates it with corresponding gesture action tags to generate data samples with gesture action tags, which are then stored in the host computer.

6. The wearable embedded gesture recognition system based on a lightweight convolutional network model and STM32 according to claim 4, characterized in that, The lightweight convolutional network model training module uses the data samples with gesture action labels stored on the host computer to select a portion of the data samples as the training group and the remaining data samples as the test group. Then, the training group data is divided into a training set and a validation set using a stratified sampling strategy to ensure that the distribution of samples of each category is balanced, thus obtaining the divided training data.

7. The wearable embedded gesture recognition system based on a lightweight convolutional network model and STM32 according to claim 6, characterized in that, The lightweight convolutional network model training module pre-trains based on the partitioned training data, constructs a network containing multiple residual blocks, sets dilated causal convolutional layers for each residual block and uses convolutional kernels of a set size, alleviates the gradient vanishing problem through residual connections, and optimizes network parameters using activation functions and optimizers combined with dynamic learning rate adjustment and gradient clipping strategies to obtain the initial gesture recognition model.

8. The wearable embedded gesture recognition system based on a lightweight convolutional network model and STM32 according to claim 7, characterized in that, The lightweight convolutional network model training module evaluates the importance of each convolutional layer channel in the initial gesture recognition model based on the L1 norm, removes channels with lower weights according to a set ratio, reconstructs a compact model, and gradually restores the model performance through progressive fine-tuning to obtain a lightweight gesture recognition model. The lightweight gesture recognition model is then transmitted to the model deployment module.

9. The wearable embedded gesture recognition system based on a lightweight convolutional network model and STM32 according to claim 8, characterized in that, The model deployment module uses a dedicated plugin and development tools to compress the lightweight gesture recognition model to reduce its size, and then converts the compressed model to be compatible with the STM32 to obtain the adapted model file. The adapted model file is then burned into the storage unit of the STM32 to realize the deployment of the lightweight gesture recognition model on the STM32.

10. The wearable embedded gesture recognition system based on a lightweight convolutional network model and STM32 according to claim 1, characterized in that, The offline real-time inference module acquires surface electromyography (EMG) signals from the middle of the user's forearm in real time through the STM32 to obtain real-time raw EMG signals. The real-time raw electromyography signal is processed according to the filtering method of the data preprocessing module to obtain real-time filtered data; The real-time filtered data is input into the lightweight gesture recognition model deployed on the STM32, the model is called to perform offline inference calculations, real-time gesture inference results are generated and transmitted to the application module; The application module parses the controlled operation command based on the real-time gesture reasoning result, generates the corresponding control signal and transmits it to the controlled device, so that the controlled device performs the corresponding action.

Images