Hand posture reconstruction method, system and equipment based on sEMG and medium

Abstract

The invention discloses a hand posture reconstruction method, system and device based on sEMG and a medium. The method comprises the following steps that a forearm sEMG signal to be estimated is collected; performing denoising processing on the forearm sEMG signal based on a continuous denoising network, and segmenting the forearm sEMG signal after denoising processing into a plurality of continuous signals; performing feature extraction on the continuous signals based on a mixed attention mechanism to obtain continuous signal features; hand posture prediction based on posture correction processing is carried out on the continuous signal features based on a dual-decoder prediction architecture, and hand continuous posture signals are obtained; calculating a hand joint angular acceleration based on the hand continuous attitude signal, and converting a hand attitude reconstruction image according to the hand joint angular acceleration; according to the method, the sEMG features are extracted more effectively by fusing sparse attention and channel attention mechanisms, the attitude correction capability is remarkably enhanced by using a double-decoder structure, and the real-time accuracy of hand attitude estimation is improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology. Specifically, the present invention is applied to the fields of biomedical engineering, human-computer interaction and rehabilitation medicine, and in particular to a hand posture reconstruction method, system, device and medium based on sEMG. Background Art

[0002] In the field of hand posture estimation technology, there are currently multiple solutions to meet the needs of different application scenarios.

[0003] First, one of the existing mainstream solutions is to process the hand image data collected by the camera. The solution first extracts fine image features from the hand image captured by the camera through high-resolution feature extraction technology; in order to understand the hand posture more comprehensively, the solution further uses heat map modules of different scales and designs three branches to learn the 3D hand joint point features, 3D hand mesh vertex features and attention features respectively. This multi-branch design helps to capture multiple key information of hand posture; based on feature extraction and learning, the solution effectively integrates multi-information features such as 3D hand joint point features and 3D hand mesh vertex features through feature fusion technology; then, the multi-layer Transformer encoder-decoder module is used to deeply learn the complex relationship between these multi-information features and global features; this deep learning mechanism helps to improve the accuracy and robustness of hand posture estimation; finally, the 3D hand joint points and 3D hand coarse mesh vertices are regressed and predicted by the regressor, and the 3D hand fine mesh vertices are generated by the mesh upsampling technology, thereby completing the task of building a 3D hand posture estimator. However, although this solution has high performance in theory, it faces some challenges in practical applications: since this solution mainly relies on video streams to analyze hand motion postures, it is easily affected by light and environmental factors; under complex lighting conditions or when the ambient noise is high, the camera may not be able to effectively collect complete hand motion information, resulting in errors in hand posture analysis; in addition, this solution is not friendly to users with hand dysfunction because they may not be able to make clear hand movements for the camera to capture.

[0004] Secondly, another hand posture estimation solution uses sensor technology. This solution collects gesture data through sensor devices such as electromyography electrodes, analog front-end chips, and multi-axis acceleration sensor modules, and uploads these data to the host computer for gesture recognition after pre-processing by the main control chip; this solution has the advantages of fast response speed and insensitivity to light and environmental factors. However, it also has some limitations: since this solution mainly uses gesture recognition for human-computer interaction, it cannot directly display complete hand movement information; in addition, when the gesture exceeds the recognition category set by the algorithm, the solution may not be able to accurately recognize the corresponding gesture.

[0005] In summary, the existing hand pose estimation schemes have their own advantages and disadvantages. Therefore, it is necessary to continuously explore new technologies and methods to overcome the limitations of existing schemes and improve the performance and applicability of hand pose estimation schemes. Summary of the invention

[0006] The object of the present invention is to provide a hand posture reconstruction method, system, device and medium based on sEMG, so as to solve all or one of the above problems existing in the prior art.

[0007] In order to solve the above technical problems, the specific technical solutions of the present invention are as follows:

[0008] In one aspect, the present invention provides a method for reconstructing hand posture based on sEMG, comprising the following steps:

[0009] Signal acquisition steps:

[0010] Collecting a forearm sEMG signal to be estimated, and determining a motion state of the forearm sEMG signal;

[0011] Signal denoising and segmentation steps:

[0012] Performing denoising processing on the forearm sEMG signal based on a continuous denoising network, and dividing the denoised forearm sEMG signal into a plurality of continuous signals;

[0013] Feature extraction and posture prediction steps:

[0014] Extracting features of the continuous signal based on a hybrid attention mechanism to obtain continuous signal features;

[0015] Based on the dual decoder prediction architecture, the continuous signal features are subjected to hand posture prediction based on posture correction processing to obtain a continuous hand posture signal;

[0016] Grid image conversion step;

[0017] The hand joint angular acceleration is calculated based on the continuous hand posture signal, and the hand posture is converted according to the hand joint angular acceleration to reconstruct the image.

[0018] As an improved solution, the denoising process of the forearm sEMG signal based on a continuous denoising network includes:

[0019] A third-order Butterworth bandpass filter is used to filter out baseline drift and high-frequency noise in the forearm sEMG signal;

[0020] A 50 Hz notch filter is used to remove power frequency interference in the forearm sEMG signal.

[0021] As an improved solution, the feature extraction of the continuous signal based on the hybrid attention mechanism to obtain the continuous signal feature includes:

[0022] After the hybrid attention mechanism acquires the continuous signal, it processes the position coding and the value coding to obtain multi-channel electromyographic coding information;

[0023] In the feature extraction process, the timing information of the continuous signal is focused on based on a multi-head sparse attention mechanism;

[0024] During the feature extraction process, the channel features of the continuous signal are focused on based on a multi-channel attention mechanism.

[0025] As an improved solution, the method of extracting features from the continuous signal based on the hybrid attention mechanism to obtain continuous signal features further includes:

[0026] Combining the outputs of the multi-head sparse attention mechanism and the multi-channel attention mechanism to obtain a hybrid attention output;

[0027] The mixed attention output is extracted into a latent space representing the hand posture using a residual network and layer regularization.

[0028] As an improved solution, the dual decoder prediction architecture includes: a preliminary posture decoder and a posture correction decoder;

[0029] The hand posture prediction based on posture correction processing includes:

[0030] Calling the preliminary gesture decoder to predict the continuous signal features to obtain a preliminary noisy gesture of the hand;

[0031] Calling the posture correction decoder to predict a posture correction range, using BiLSTM to correct the preliminary noisy posture according to the predicted posture correction range, and outputting a correction result;

[0032] The correction result is used as the hand continuous gesture signal.

[0033] As an improved solution, the method of calculating the hand joint angular acceleration based on the hand continuous posture signal and converting the hand posture to reconstruct the image according to the hand joint angular acceleration includes:

[0034] The PD controller is used to convert the continuous hand gesture signal into corresponding joint torque and joint angular acceleration;

[0035] The Mano hand model is called to generate a hand posture reconstruction image about the hand continuous posture signal according to the joint angular acceleration and the hand joint posture.

[0036] As an improved solution, before the signal acquisition step, the method further includes:

[0037] Collecting eight-channel sEMG signal data, and performing posture annotation on the eight-channel sEMG signal data to obtain a training data set;

[0038] The joint optimization strategy of MSE loss and KL divergence loss is adopted as the joint loss function;

[0039] Constructing a pose estimation model based on the hybrid attention mechanism and the dual decoder prediction architecture;

[0040] The training data set and the joint loss function are used to perform supervised learning training on the posture estimation model, optimize the loss function, and optimize the model parameters of the posture estimation model.

[0041] On the other hand, the present invention also provides a hand posture reconstruction system based on sEMG, comprising:

[0042] A signal acquisition module, used for acquiring a forearm sEMG signal to be estimated and determining a motion state of the forearm sEMG signal;

[0043] A signal denoising and segmentation module, used for denoising the forearm sEMG signal based on a continuous denoising network, and segmenting the denoised forearm sEMG signal into a plurality of continuous signals;

[0044] A feature extraction and posture prediction module, which is used to extract features from the continuous signal based on a hybrid attention mechanism to obtain continuous signal features; and to perform hand posture prediction based on posture correction processing on the continuous signal features based on a dual decoder prediction architecture to obtain a continuous hand posture signal;

[0045] The grid image conversion module is used to calculate the hand joint angular acceleration based on the hand continuous posture signal, and reconstruct the hand posture image according to the hand joint angular acceleration conversion.

[0046] On the other hand, the present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the sEMG-based hand posture reconstruction method are implemented.

[0047] On the other hand, the present invention further provides a computer device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; wherein:

[0048] The memory is used to store computer programs;

[0049] The processor is used to execute the steps of the sEMG-based hand posture reconstruction method by running the program stored in the memory.

[0050] The beneficial effects of the technical solution of the present invention are:

[0051] 1. The sEMG-based hand posture reconstruction method described in the present invention can extract sEMG features more effectively by fusing sparse attention and channel attention mechanisms, and at the same time significantly enhance the posture correction capability by utilizing a dual decoder structure, thereby achieving higher estimation accuracy; by optimizing the algorithm and model structure, real-time hand posture estimation is achieved, meeting the demand for rapid response in practical applications; a modular design is adopted, so that the functions of each module are easy to modify and expand; in addition, the present invention can easily adjust the module configuration according to actual needs, such as replacing the feature extraction model or the posture estimator, thereby improving flexibility and customizability; at the same time, the present invention has broad application potential in multiple fields such as virtual reality, gesture control, and rehabilitation training, and can provide users with a more accurate and real-time hand posture estimation solution, making up for the defects of the prior art and having a high application value.

[0052] 2. The sEMG-based hand posture reconstruction system described in the present invention can realize the sEMG-based hand posture reconstruction method described in the present invention through the mutual cooperation of system modules.

[0053] 3. The computer-readable storage medium described in the present invention can realize the cooperation of the guidance system module, thereby realizing the sEMG-based hand posture reconstruction method described in the present invention, and the computer-readable storage medium described in the present invention also effectively improves the operability of the sEMG-based hand posture reconstruction method.

[0054] 4. The computer device described in the present invention can store and execute the computer-readable storage medium, thereby realizing the sEMG-based hand posture reconstruction method described in the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0056] Figure 1 is a flowchart of the hand posture reconstruction method based on sEMG according to Example 1 of the present invention;

[0057] Figure 2 is a detailed flowchart of the hand posture reconstruction method based on sEMG described in Example 1 of the present invention;

[0058] Figure 3 Schematic diagram of the architecture of the feature extraction module in the hand posture reconstruction method based on sEMG according to Example 1 of the present invention;

[0059] Figure 4 is a schematic diagram of the architecture of the hand posture reconstruction system based on sEMG according to Embodiment 2 of the present invention;

[0060] Figure 5 is a schematic diagram of the architecture of the hand posture reconstruction system based on sEMG in practical application according to Example 2 of the present invention;

[0061] Figure 6 is a schematic diagram of the structure of the computer device described in Example 4 of the present invention;

[0062] The symbols in the accompanying drawings are explained as follows:

[0063] 1501, processor; 1502, communication interface; 1503, memory; 1504, communication bus. DETAILED DESCRIPTION

[0064] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more definite definition of the protection scope of the present invention.

[0065] In the description of the present invention, it should be noted that the embodiments described in the present invention are only part of the embodiments of the present invention, rather than all of the embodiments; based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present invention.

[0066] The terms "first", "second", etc. in the specification and claims of this article and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of this article described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0067] In the description of the present invention, it should be noted that: sEMG is a surface electromyography signal; BiLSTM is a bidirectional long short-term memory network; Mano is a 3D hand parameterized model.

[0068] Example 1

[0069] This embodiment provides a hand posture reconstruction method based on sEMG. Figure 1 to Figure 3 As shown, the following steps are included:

[0070] S100, data collection and labeling steps, including:

[0071] S101, collecting forearm electromyographic signals using an eight-channel electromyographic collector;

[0072] S102, labeling the collected electromyographic data, wherein the collected electromyographic data are labeled as different hand postures or specific hand movements; different hand postures include: still, clenched fist, stretching, etc.;

[0073] Through this step, a basic data set is constructed and the data is labeled for subsequent training of the posture estimation model. During subsequent training, the input of eight-channel sEMG signals can analyze the movements of more joints (such as 15 hand joints), which is more comprehensive than existing solutions.

[0074] S200, signal denoising and segmentation step, including:

[0075] S201, use a third-order Butterworth bandpass filter (20-200 Hz) to filter out baseline drift and high-frequency noise in the sEMG signal;

[0076] S202, using a 50 Hz notch filter to remove power frequency interference in the sEMG signal;

[0077] S203, dividing the sEMG signal after the above processing into a plurality of overlapping sliding windows of fixed length;

[0078] Through this step, the continuous denoising network is used to pre-process the electromyographic signals collected and processed by S100 to remove noise and interference, improve data accuracy, and prepare for the subsequent feature extraction step.

[0079] S300, feature extraction and posture prediction steps, including:

[0080] S301, inputting the segmented sEMG signal into a feature extraction module for feature extraction;

[0081] In a preferred embodiment, the feature extraction module adopts a deep learning model that combines a multi-head sparse attention mechanism and a multi-channel attention mechanism; through this design, the model can focus on the correlation between different channels and comprehensively consider the synergy of different muscles, thereby improving the effectiveness of feature representation;

[0082] In a preferred embodiment, based on the above hybrid attention mechanism, eight-channel forearm electromyographic signals are obtained. After that, multi-channel electromyographic coding information is obtained through position coding and value coding. The specific formula is as follows. The multi-head sparse self-attention mechanism only saves the attention weights with greater influence, and other attention is replaced by average attention:

[0083]

[0084] In this formula: M refers to the attention that will be saved, refers to the attention that will be replaced, i refers to the dimension of the index, j refers to the number of layers of the current attention calculation, Top refers to selecting the maximum attention value, k refers to the attention value of a dimension, and L k is the length of k.

[0085] In a preferred implementation, the multi-head sparse attention mechanism, in the process of layer-by-layer superposition, the attention with greater influence will be redundant, affecting the calculation efficiency; the specific formula is as follows: In this mechanism, by jt After applying the one-dimensional convolution operation, the ELU activation function is applied, and finally the feature map of the j+1th layer is obtained through the maximum pooling operation:

[0086]

[0087] In this formula: Conv1D is a one-dimensional convolutional network, ELU is an activation function, MaxPool is a maximum pooling layer; t refers to the time point of the time series; this formula is used to assign the attention purification of layer j to layer j+1.

[0088] In a preferred implementation, the multi-channel attention mechanism focuses on important channel features, assigns different weight parameters to different channels, and multiplies the assigned weight parameters with the channel values ​​to effectively distinguish important channels from unimportant channels. The specific formula is as follows:

[0089]

[0090] In this formula: s c Refers to the channel weight, U c Refers to the channel value of channel c, where c represents the channel number. Refers to the dot product of weight and channel value; w1 and w2 represent the weight matrices of the two fully connected layers, δ and σ represent the ReLU activation function and the Sigmoid activation function, respectively; H refers to the length of the input matrix of the c channel, and W refers to the width of the input matrix of the c channel; z c Refers to the activation parameters of the channel.

[0091] In a preferred embodiment, the outputs of the multi-head sparse attention mechanism and the multi-channel attention mechanism are combined, and the residual network and layer regularization are used to extract the mixed attention mechanism output into a latent space Z representing the hand posture. The specific formula is as follows:

[0092]

[0093] In this formula: Linear represents the linear layer, E is the multi-channel electromyography encoding information, and LayerNorm refers to layer normalization.

[0094] Through this step, the sparse attention mechanism is used to focus on important timing information, avoid redundant calculations, reduce computing resource usage, and improve computing efficiency; the multi-channel attention mechanism is used to focus on important channel features, improve feature extraction efficiency and feature extraction value; based on the combination of the above two attention mechanisms, the spatiotemporal features contained in the sEMG signal are effectively extracted.

[0095] S302, sending the extracted features to a posture prediction module of a dual decoder structure for posture prediction;

[0096] In a preferred embodiment, the dual decoder structure is composed of a preliminary posture decoder and a posture correction decoder;

[0097] In a preferred embodiment, a preliminary posture decoder is called to predict and obtain a preliminary noisy posture of the hand; the input of the preliminary posture decoder is empty in the first frame, and the subsequent frames are the posture predicted in the previous frame;

[0098] In a preferred embodiment, a posture correction decoder is called to predict a correction range of posture, which uses a bidirectional long short-term memory network BiLSTM to learn prior knowledge within the posture correction range, extract important time series features in the sEMG signal, and correct the preliminary noisy posture predicted by the preliminary posture decoder according to the predicted posture correction range, so as to further improve the estimation accuracy;

[0099] Through this step, the noisy posture and the posture correction range are analyzed separately based on the dual decoder structure, and then the noisy posture is further corrected according to the posture correction range to improve the accuracy of hand posture estimation.

[0100] S303. Finally output the predicted continuous hand gesture signal.

[0101] S400, a grid image conversion step, comprising:

[0102] S401, using a single PD controller to convert the aforementioned continuous hand gesture signal into corresponding joint torque and joint angular acceleration;

[0103] S402, inputting the joint angular acceleration into the Mano model, calling the Mano hand model to generate a corresponding grid image according to the hand joint posture;

[0104] In a preferred embodiment, the desired joint angular acceleration a is calculated based on the joint rotation angle θ. des , the specific formula is as follows:

[0105]

[0106] In this formula: K p Refers to the proportional controller, K d refers to the differential controller, E(θ) is the joint rotation angle obtained by the model, a des is the expected joint angular acceleration that needs to be calculated; is the differential of θ, i.e. the joint angular velocity; θ0 is the joint angle.

[0107] Through this step, the real-time hand posture after reconstruction can be displayed intuitively and visually.

[0108] S500, model optimization step, including:

[0109] S501, defining the loss function of the hand posture estimation network, and adopting a joint optimization strategy of MSE loss and KL divergence loss; wherein MSE (mean square error) is used to measure the difference between the predicted posture and the true posture; KL divergence loss is used to optimize the crosstalk problem caused by multi-finger linkage;

[0110] S502, by minimizing the loss function, the entire posture estimation model (feature extraction module + posture prediction module) of S300 is subjected to supervised learning training, the loss function is optimized, and the model parameters are continuously optimized to improve the estimation accuracy of the posture estimation model;

[0111] In a preferred embodiment, the loss function is:

[0112]

[0113] In this formula: and p r Represent the predicted value and the true value of the attitude range, respectively, which are output by two decoders in S300; P * and P represent the predicted value and true value of the posture respectively. The predicted value is output by BiLSTM in S300; KL refers to KL divergence loss, MSE refers to MSE loss, and m refers to the number of predicted values.

[0114] Through this step, the joint optimization strategy based on MSE loss and KL divergence loss can effectively reduce noise and error accumulation and improve estimation accuracy.

[0115] In summary, the three-dimensional hand posture estimation scheme of the present invention can collect forearm electromyographic signals, simulate hand movement training in a virtual environment, and reconstruct the complete movement structure of the hand in three-dimensional space based on hand gesture estimation technology, thereby assisting in the remodeling of the nervous system and functional recovery, and has high application value.

[0116] It should be noted that all the above examples are only for explaining the present invention and cannot limit the protection scope of the present invention.

[0117] Example 2

[0118] This embodiment is based on the same inventive concept as the hand posture reconstruction method based on sEMG described in Embodiment 1, and provides a hand posture reconstruction system based on sEMG, such as Figure 4 and Figure 5 As shown, including:

[0119] A signal acquisition module, used for acquiring a forearm sEMG signal to be estimated and determining a motion state of the forearm sEMG signal;

[0120] A signal denoising and segmentation module, used for denoising the forearm sEMG signal based on a continuous denoising network, and segmenting the denoised forearm sEMG signal into a plurality of continuous signals;

[0121] A feature extraction and posture prediction module, which is used to extract features from the continuous signal based on a hybrid attention mechanism to obtain continuous signal features; and to perform hand posture prediction based on posture correction processing on the continuous signal features based on a dual decoder prediction architecture to obtain a continuous hand posture signal;

[0122] The grid image conversion module is used to calculate the hand joint angular acceleration based on the hand continuous posture signal, and reconstruct the hand posture image according to the hand joint angular acceleration conversion.

[0123] As a preferred implementation method, when the above macroscopic system architecture is actually applied, the system is specifically manifested as two parts: an eight-channel myoelectric collector and a host computer system. The specific functional logic of these two parts is as follows:

[0124] (1) An eight-channel myoelectric collector, comprising: an acquisition module, a main control module, a preprocessing module and a Bluetooth module; the acquisition module is provided with an electromyographic AD acquisition module, a multi-axis gyroscope acquisition module and a signal amplifier;

[0125] The acquisition module is mainly used to collect the myoelectric signals and posture signals of the forearm and obtain the corresponding digital signals through the cooperation of the myoelectric AD acquisition module, the multi-axis gyroscope acquisition module and the signal amplifier. The acquisition frequency is 450hz.

[0126] The main control module is mainly used to coordinate the component collaboration of multiple acquisition modules;

[0127] The preprocessing module is mainly used to divide the electromyographic data into rest-exercise-rest according to the posture data and the electromyographic data;

[0128] The Bluetooth module is connected to the host computer and is mainly used to transmit the collected electromyographic signals to the host computer system.

[0129] (2) Host computer system, including electromyography receiving module, hand posture estimation module and hand grid demonstration module;

[0130] The electromyography receiving module is connected to the aforementioned eight-channel electromyography collector via Bluetooth, and is mainly used to display the waveform of the eight-channel electromyography signal collected on the host computer;

[0131] The hand posture estimation module includes an end-to-end algorithm unit and a controller unit; wherein the end-to-end algorithm unit is used to convert the collected electromyographic signals into discrete hand posture signals; the controller unit is used to convert the discrete hand posture signals into joint rotation acceleration and corresponding torque by using PD control;

[0132] The hand grid demonstration module is used to use the model to construct continuous hand gesture signals into a grid image for demonstration in the interface.

[0133] (3) In addition, it also includes: a lower computer system; the lower computer system is mainly called by the eight-channel electromyography collector to complete the operations of data collection and labeling, signal preprocessing and segmentation in Example 1.

[0134] Example 3

[0135] This embodiment provides a computer-readable storage medium, including:

[0136] The storage medium is used to store computer software instructions used to implement the sEMG-based hand posture reconstruction method described in the above-mentioned Example 1, which includes instructions for executing the above-mentioned program set for the sEMG-based hand posture reconstruction method; specifically, the executable program can be built into the sEMG-based hand posture reconstruction system described in Example 2, so that the sEMG-based hand posture reconstruction system can implement the sEMG-based hand posture reconstruction method described in Example 1 by executing the built-in executable program.

[0137] In addition, the computer-readable storage medium of this embodiment may adopt any combination of one or more computer-readable storage media, wherein the computer-readable storage medium includes electrical, optical, electromagnetic, infrared or semiconductor systems, devices or components, or any combination thereof.

[0138] Example 4

[0139] This embodiment provides an electronic device, such as Figure 6 As shown, the electronic device may include: a processor 1501 , a communication interface 1502 , a memory 1503 and a communication bus 1504 , wherein the processor 1501 , the communication interface 1502 , and the memory 1503 communicate with each other via the communication bus 1504 .

[0140] Memory 1503, used for storing computer programs;

[0141] The processor 1501 is used to implement the steps of the hand posture reconstruction method based on sEMG described in the above embodiment 1 when executing the computer program stored in the memory 1503.

[0142] As an embodiment of the present invention, the communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0143] As an implementation mode of the present invention, the communication interface is used for communication between the above-mentioned terminal and other devices.

[0144] As an embodiment of the present invention, the memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0145] As an embodiment of the present invention, the above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0146] Different from the prior art, the present invention adopts a sEMG-based hand posture reconstruction method, system, device and medium, which can more effectively extract sEMG features by fusing sparse attention and channel attention mechanisms, and significantly enhance the posture correction capability by utilizing a dual decoder structure, thereby achieving higher estimation accuracy; through the optimization of the algorithm and model structure, real-time hand posture estimation is achieved, meeting the demand for rapid response in practical applications; a modular design is adopted, so that the functions of each module are easy to modify and expand; in addition, the present invention can easily adjust the module configuration according to actual needs, such as replacing the feature extraction model or the posture estimator, thereby improving flexibility and customizability; at the same time, the present invention has broad application potential in multiple fields such as virtual reality, gesture control and rehabilitation training, and can provide users with a more accurate and real-time hand posture estimation solution, making up for the defects of the prior art and having a high application value.

[0147] It should be understood that in the various embodiments of this document, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this document.

[0148] It should also be understood that in the embodiments of this article, the term "and / or" is only a description of the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0149] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this article.

[0150] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0151] In the several embodiments provided herein, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or can be electrical, mechanical or other forms of connection.

[0152] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of this article.

[0153] In addition, each functional unit in each embodiment of this invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional unit.

[0154] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this article is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of this article. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0155] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A hand posture reconstruction method based on sEMG, characterized in that: The following steps are involved: Signal acquisition steps: Collecting a forearm sEMG signal to be estimated, and determining a motion state of the forearm sEMG signal; Signal denoising and segmentation steps: Performing denoising on the forearm sEMG signal based on a continuous denoising network, and dividing the denoised forearm sEMG signal into a plurality of continuous signals; Feature extraction and posture prediction steps: Extracting features of the continuous signal based on a hybrid attention mechanism to obtain continuous signal features; Based on the dual decoder prediction architecture, the continuous signal features are subjected to hand posture prediction based on posture correction processing to obtain a continuous hand posture signal; Grid image conversion step; The hand joint angular acceleration is calculated based on the continuous hand posture signal, and the hand posture is converted according to the hand joint angular acceleration to reconstruct the image.

2. The hand posture reconstruction method based on sEMG according to claim 1, characterized in that: The denoising process of the forearm sEMG signal based on a continuous denoising network comprises: A third-order Butterworth bandpass filter is used to filter out baseline drift and high-frequency noise in the forearm sEMG signal; A 50 Hz notch filter is used to remove power frequency interference in the forearm sEMG signal.

3. The hand posture reconstruction method based on sEMG according to claim 1, characterized in that: The step of extracting features from the continuous signal based on the hybrid attention mechanism to obtain continuous signal features includes: After the hybrid attention mechanism acquires the continuous signal, it processes the position coding and the value coding to obtain multi-channel electromyographic coding information; In the feature extraction process, the timing information of the continuous signal is focused on based on a multi-head sparse attention mechanism; During the feature extraction process, the channel features of the continuous signal are focused on based on a multi-channel attention mechanism.

4. The hand posture reconstruction method based on sEMG according to claim 3, characterized in that: The extracting features of the continuous signal based on the hybrid attention mechanism to obtain the continuous signal features also includes: Combining the outputs of the multi-head sparse attention mechanism and the multi-channel attention mechanism to obtain a hybrid attention output; The mixed attention output is extracted into a latent space representing the hand posture using a residual network and layer regularization.

5. The hand posture reconstruction method based on sEMG according to claim 1, characterized in that: The dual decoder prediction architecture includes: a preliminary posture decoder and a posture correction decoder; The hand posture prediction based on posture correction processing includes: Calling the preliminary gesture decoder to predict the continuous signal features to obtain a preliminary noisy gesture of the hand; Calling the posture correction decoder to predict a posture correction range, using BiLSTM to correct the preliminary noisy posture according to the predicted posture correction range, and outputting a correction result; The correction result is used as the hand continuous gesture signal.

6. The hand posture reconstruction method based on sEMG according to claim 1, characterized in that: The method of calculating the hand joint angular acceleration based on the hand continuous posture signal and converting the hand posture to reconstruct the image according to the hand joint angular acceleration includes: The PD controller is used to convert the continuous hand gesture signal into corresponding joint torque and joint angular acceleration; The Mano hand model is called to generate a hand posture reconstruction image about the hand continuous posture signal according to the joint angular acceleration and the hand joint posture.

7. The hand posture reconstruction method based on sEMG according to claim 1, characterized in that: Before the signal acquisition step, it also includes: Collecting eight-channel sEMG signal data, and performing posture annotation on the eight-channel sEMG signal data to obtain a training data set; The joint optimization strategy of MSE loss and KL divergence loss is adopted as the joint loss function; Constructing a pose estimation model based on the hybrid attention mechanism and the dual decoder prediction architecture; The training data set and the joint loss function are used to perform supervised learning training on the posture estimation model, optimize the loss function, and optimize the model parameters of the posture estimation model.

8. A hand posture reconstruction system based on sEMG, characterized in that: include: A signal acquisition module, used for acquiring a forearm sEMG signal to be estimated and determining a motion state of the forearm sEMG signal; A signal denoising and segmentation module, used for denoising the forearm sEMG signal based on a continuous denoising network, and segmenting the denoised forearm sEMG signal into a plurality of continuous signals; A feature extraction and posture prediction module, which is used to extract features from the continuous signal based on a hybrid attention mechanism to obtain continuous signal features; and to perform hand posture prediction based on posture correction processing on the continuous signal features based on a dual decoder prediction architecture to obtain a continuous hand posture signal; The grid image conversion module is used to calculate the hand joint angular acceleration based on the hand continuous posture signal, and reconstruct the hand posture image according to the hand joint angular acceleration conversion.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the hand posture reconstruction method based on sEMG in any one of claims 1 to 7 are implemented.

10. A computer device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; wherein: The memory is used to store computer programs; The processor is used to execute the steps of the sEMG-based hand posture reconstruction method according to any one of claims 1 to 7 by running the program stored in the memory.

Images