A method and system for myoelectric signal segmentation for continuous operation sequences
By standardizing and preprocessing electromyographic signals, performing semantic-level segmentation, and detecting the transfer relationships between actions, the final predicted action sequence is generated. This solves the problem of analyzing continuous operation sequences in existing technologies and enables effective segmentation and recognition of operations in natural states.
Patent Information
- Application Number
- CN202310041430.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-01-11
- Publication Date
- 2026-01-23
- Estimated Expiration
- 2043-01-11
AI Technical Summary
Existing electromyography (EMG) signal analysis methods are insufficient for effectively analyzing continuous operation sequences of test subjects in the time dimension, and cannot accurately segment and identify operation behaviors in a natural state.
An electromyography (EMG) signal segmentation method is adopted, including standardized preprocessing, semantic-level segmentation, detection and optimization integration of inter-action transition relationships. The EMG signal is segmented and identified using a U-net-like structure to generate the final predicted action sequence.
It achieves the segmentation of electromyographic signals in the time dimension, outputs basic operational units, supports the analysis of the operation execution of test subjects in a natural state, breaks the limitations of the traditional turn-based system, and improves the ability to analyze continuous operation tasks.
Smart Images
Figure CN116204767B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of physiological signal processing, and in particular to an electromyography signal segmentation method and system for continuous operation sequences. BACKGROUND
[0002] Analysis of human subject's operation skills using electrophysiological signals has always been widely concerned, and common electrophysiological signals include electroencephalogram and electromyography. Compared with electroencephalogram, electromyography has advantages such as stability, low cost and direct reflection of muscle activity, and therefore is more popular in practical applications. However, current electromyography-based motion analysis researches are mostly based on round-based tasks, that is, a tester is given a prompt, then performs a specific operation in a specified time period, and then an intelligent algorithm is used to analyze electromyography recorded during the operation. However, the tester's operation execution in this experimental paradigm is seriously inconsistent with the natural state. In the natural state, the tester performs operations continuously, although the operation sequence is composed of basic operation units, these operations are not executed discretely, and the tester's operation rhythm is different in different test times, which is quite different from the operation research within a limited time range. Based on these current situations, if a method for analyzing the continuous operation of subjects in the natural state can be designed, it will help researchers have a deeper understanding of human operation behavior.
[0003] In the prior art, the completion degree of the tester's motion and the current fatigue state are analyzed, or a wearable motion analysis device is used to analyze the tester in the activity. However, neither the device nor the method can analyze the tester's electromyography sequence operation behavior in the time dimension. SUMMARY
[0004] The present application provides an electromyography signal segmentation method and system for continuous operation sequences to solve the problem that existing electromyography signals are difficult to analyze sequence operation behavior.
[0005] The present application provides an electromyography signal segmentation method for continuous operation sequences, comprising:
[0006] Obtaining electromyography signals, and performing standardization preprocessing on the electromyography signals;
[0007] Performing semantic-level segmentation processing on the electromyography signals after standardization preprocessing to generate a segmentation result;
[0008] Performing recognition and detection of the transfer relationship between actions on the electromyography signals after standardization preprocessing to generate a transfer result;
[0009] The segmentation result and the transfer result are optimized, part of the counterfactual and abnormal detection result is removed, integrated output is carried out, and a final predicted action sequence is obtained.
[0010] According to the myoelectric signal segmentation method for continuous operation sequence provided by the application, the myoelectric signal is obtained, and the myoelectric signal is standardized and pretreated, specifically including:
[0011] The original myoelectric signal is obtained, and the myoelectric signal is decentered and standardized and pretreated, and the processing process is:
[0012]
[0013] Wherein E represents the myoelectric signal, ave represents the mean operation, and std represents the standard deviation operation.
[0014] According to the myoelectric signal segmentation method for continuous operation sequence provided by the application, the myoelectric signal after standardization pretreatment is subjected to semantic level segmentation processing, and segmentation result is generated, specifically including:
[0015] The myoelectric signal after standardization pretreatment is input into a preset semantic segmentation model, and the operation category corresponding to each time point is output after processing;
[0016] The myoelectric signal in the operation process is classified and processed at the myoelectric time resolution level through the U-net structure, and the segmentation result is formed.
[0017] According to the myoelectric signal segmentation method for continuous operation sequence provided by the application, the myoelectric signal after standardization pretreatment is subjected to recognition and detection of the transfer relationship between actions, and the transfer result is generated, specifically including:
[0018] The myoelectric signal after standardization pretreatment is input into a preset transfer time detection model, and whether the action changes at the current time point is output;
[0019] The switching time between actions is detected and recognized through the U-net structure consistent with the semantic segmentation module, and the transfer result is generated.
[0020] According to the myoelectric signal segmentation method for continuous operation sequence provided by the application, the segmentation result and the transfer result are optimized, part of the counterfactual and abnormal detection result is removed, specifically including:
[0021] The segmentation result and the transfer result are input into a preset action sequence optimization model, and the action sequence optimization model is optimized to output a sequence action group;
[0022] The sequence action group comprises an action category and a start time, and the detection results of partial counterfactuals and abnormalities are removed.
[0023] The application provides an electromyographic signal segmentation method for a continuous operation sequence.
[0024] The action with the highest action transition time interval occurrence probability in the detection results of the partial counterfactuals and abnormalities is taken as the action in the current time period.
[0025] The actions in different time periods are integrated to obtain a final predicted action sequence.
[0026] The application further provides an electromyographic signal segmentation system for a continuous operation sequence.
[0027] A data preprocessing module is configured to acquire electromyographic signals and perform standardization preprocessing on the electromyographic signals.
[0028] An operation semantic segmentation module is configured to perform semantic-level segmentation processing on the electromyographic signals after standardization preprocessing to generate a segmentation result.
[0029] A transition time detection module is configured to identify and detect the transition relationship between actions for the electromyographic signals after standardization preprocessing to generate a transition result.
[0030] An action sequence optimization module is configured to optimize the segmentation result and the transition result to remove the detection results of partial counterfactuals and abnormalities.
[0031] A result output module is configured to perform integrated output to obtain a final predicted action sequence.
[0032] The application further provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the electromyographic signal segmentation method for a continuous operation sequence as described above.
[0033] The application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program is executable on a processor to implement the electromyographic signal segmentation method for a continuous operation sequence as described above.
[0034] The application further provides a computer program product including a computer program, wherein the computer program is executable on a processor to implement the electromyographic signal segmentation method for a continuous operation sequence as described above.
[0035] The application provides a myoelectric signal segmentation method and system for continuous operation sequences, which segments myoelectric signals in the time dimension, outputs operation basic units corresponding to the myoelectric signals at different operation times, and provides a basic support for operation execution of a tester in a natural state. BRIEF DESCRIPTION OF DRAWINGS
[0036] In order to more clearly illustrate the technical solutions in the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.
[0037] Figure 1 is one of the flowcharts of the myoelectric signal segmentation method for continuous operation sequences provided by the application;
[0038] Figure 2 is the second flowchart of the myoelectric signal segmentation method for continuous operation sequences provided by the application;
[0039] Figure 3 is the third flowchart of the myoelectric signal segmentation method for continuous operation sequences provided by the application;
[0040] Figure 4 is the fourth flowchart of the myoelectric signal segmentation method for continuous operation sequences provided by the application;
[0041] Figure 5 is the module connection diagram of the myoelectric signal segmentation system for continuous operation sequences provided by the application;
[0042] Figure 6 is the U-net network structure diagram in the operation semantic segmentation module and the transfer time detection module provided by the application;
[0043] Figure 7 is the structural diagram of the electronic device provided by the application.
[0044] Corresponding reference signs:
[0045] 110: data preprocessing module; 120: operation semantic segmentation module; 130: transfer time detection module; 140: action sequence optimization module; 150: result output module;
[0046] 710: processor; 720: communication interface; 730: memory; 740: communication bus. DETAILED DESCRIPTION
[0047] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative labor fall within the protection scope of the present application.
[0048] The present application is described below with reference to the drawings Figures 1-4 The present application is described below with reference to the drawings
[0049] S100, acquiring an electromyography signal, and performing standardized preprocessing on the electromyography signal;
[0050] S200, performing semantic-level segmentation processing on the electromyography signal after the standardized preprocessing, to generate a segmentation result;
[0051] S300, performing identification and detection of a transition relationship between actions on the electromyography signal after the standardized preprocessing, to generate a transition result;
[0052] S400, optimizing the segmentation result and the transition result, removing part of counterfactual and abnormal detection results, integrating and outputting, to obtain a final predicted action sequence.
[0053] In the present application, a method for analyzing electromyography signals of a subject during continuous operation execution can segment the electromyography signals in the time dimension, output the operation basic units corresponding to the electromyography signals at different operation times, and lay a foundation for evaluating the operation execution of the subject in a natural state.
[0054] The acquiring of the electromyography signal and the standardized preprocessing of the electromyography signal specifically include:
[0055] The acquiring of the original electromyography signal and the decentering and standardized preprocessing of the electromyography signal specifically include:
[0056]
[0057] Wherein E represents the electromyography signal, ave represents the mean operation, and std represents the standard deviation operation. The processing of the original electromyography signal can reduce errors.
[0058] The semantic-level segmentation processing on the electromyography signal after the standardized preprocessing to generate a segmentation result specifically includes:
[0059] S201, input the standardized preprocessed electromyographic signal to a preset semantic segmentation model, and output an operation category corresponding to each time point after processing;
[0060] S202, classify and process the electromyographic signal in the operation process through a U-net-like structure to form a segmentation result.
[0061] The standardized preprocessed electromyographic signal is recognized and detected for the transfer relationship between actions, and a transfer result is generated, specifically including:
[0062] S301, input the standardized preprocessed electromyographic signal to a preset transfer time detection model, and output whether an action change occurs at the current time point;
[0063] S302, detect and recognize the switching time between actions through a U-net-like structure consistent with the semantic segmentation module to generate a transfer result.
[0064] In the present application, the semantic segmentation model and the transfer time detection model have a network structure similar to U-net, as shown in Figure 6 The final activation is performed using a softmax function, and the output is performed according to the classification category. The two networks are trained using a cross-entropy loss function and a focal loss, respectively.
[0065] The U-net-like structure is derived from the U-net structure which has achieved great success in image semantic segmentation tasks, and some changes are made to make it more suitable for processing electromyographic signals. In the encoding process, the two two-dimensional convolutions in the original U-net are replaced by one one-dimensional convolution and one depth separable convolution, which reduces the operation parameters and better excavates the space-time correlation between signals; in the encoding layer, the original two-dimensional convolution is replaced by a one-dimensional convolution, which further focuses on the correlation of features in the time dimension.
[0066] The segmentation result and the transfer result are optimized to remove part of the counterfactual and abnormal detection results, specifically including:
[0067] S401, input the segmentation result and the transfer result to a preset action sequence optimization model, and output a sequence action group after optimization processing by the action sequence optimization model.
[0068] S402, the sequence action group includes an action category and a start time, and part of the counterfactual and abnormal detection results are removed.
[0069] The input of the action sequence optimization module is the output y seg of the operation semantic segmentation module and the output y trans, the output is a sequence of action groups, including action types and start time. The specific steps are as follows.
[0070] First, the output y of the transition time detection module trans The state transition point is distinguished by a dynamic threshold:
[0071] y trans ←[y trans >ave(y trans )+std(y trans )]
[0072] Then find the start time point and end time point of the action transition in y trans according to the results of the above formula. Then traverse y trans , the action transition time that cannot be continuous is determined to be noise processing; at the same time, the state transition with an action interval less than 1s is taken as the average clustering processing, and finally the obtained results are taken into the output y seg of the semantic segmentation module, the action with the highest occurrence probability between adjacent action transition time intervals is taken as the action in this time period.
[0073] Integrate the output to obtain the final predicted action sequence, which specifically includes:
[0074] The action with the highest occurrence probability between adjacent action transition time intervals in the detection result after removing part of the counterfactual and anomaly is taken as the action in the current time period;
[0075] Integrate the actions in different time periods to obtain the final predicted action sequence.
[0076] The myoelectric signal segmentation method for continuous operation sequence provided by the application segments the myoelectric signal in the time dimension, outputs the operation basic unit corresponding to the myoelectric signal at different operation times, and provides a basic support for operation execution of testers in a natural state. The algorithm for automatically segmenting and identifying the myoelectric signal in the continuous operation task breaks the requirement of sequence round in traditional human-computer interaction tasks, and is beneficial to analyzing the operation mode of the tester under the operation logic more in line with nature.
[0077] Reference Figure 5 , the application also discloses a myoelectric signal segmentation system for continuous operation sequence, which comprises:
[0078] A data preprocessing module 110 is used to acquire myoelectric signals and perform standardization preprocessing on the myoelectric signals.
[0079] An operation semantic segmentation module 120 is used to perform semantic-level segmentation processing on the myoelectric signals after standardization preprocessing, and generate a segmentation result.
[0080] The transfer time detection module 130 is configured to identify and detect the transfer relationship between the action sequences in the normalized preprocessed electromyography signals, and generate a transfer result.
[0081] The action sequence optimization module 140 is configured to optimize the segmentation result and the transfer result, and remove part of the counterfactual and abnormal detection results.
[0082] The result output module 150 is configured to perform integrated output to obtain a final predicted action sequence.
[0083] The data preprocessing module 110 is configured to obtain original electromyography signals, and perform decentralization and standardization preprocessing on the electromyography signals, and the processing process is as follows:
[0084]
[0085] Wherein E represents electromyography signals, ave represents mean operation, and std represents standard deviation operation.
[0086] The operation semantic segmentation module 120 is configured to input the normalized preprocessed electromyography signals into a preset semantic segmentation model, and output the operation category corresponding to each time point after processing.
[0087] The electromyography signals in the operation process are classified and processed by the U-net structure to form a segmentation result.
[0088] The transfer time detection module 130 is configured to input the normalized preprocessed electromyography signals into a preset transfer time detection model, and output whether the action changes at the current time point.
[0089] The switching time between actions is detected and identified by the U-net structure consistent with the semantic segmentation module to generate a transfer result.
[0090] The action sequence optimization module 140 is configured to input the segmentation result and the transfer result into a preset action sequence optimization model, and output a sequence action group after optimization processing by the action sequence optimization model.
[0091] The sequence action group includes action categories and starting times, and part of the counterfactual and abnormal detection results are removed.
[0092] The result output module 150 is configured to take the action with the highest action transfer time interval occurrence probability in the detection result after removing part of the counterfactual and abnormal detection results as the action in the current time period.
[0093] The actions in different time periods are integrated to obtain a final predicted action sequence.
[0094] The myoelectric signal segmentation system for continuous operation sequence provided by the application segments myoelectric signals in the time dimension, outputs operation basic units corresponding to myoelectric signals at different operation times, and provides a basis for operation execution of testers in a natural state. The algorithm for automatically segmenting and identifying muscle electric signals in continuous operation tasks breaks the requirement for sequence rounds in traditional human-computer interaction tasks, and is beneficial to analyzing the operation mode of testers in a natural operation logic.
[0095] Figure 7 An example of an entity structure diagram of an electronic device is shown in Figure 7 The electronic device can include a processor 710, a communications interface 720, a memory 730, and a communications bus 740, wherein the processor 710, the communications interface 720, and the memory 730 communicate with each other through the communications bus 740. The processor 710 can invoke a logical instruction in the memory 730 to execute a myoelectric signal segmentation method for a continuous operation sequence, which includes: acquiring a myoelectric signal, and performing standardization preprocessing on the myoelectric signal;
[0096] The standardization preprocessed myoelectric signal is subjected to semantic level segmentation processing to generate a segmentation result;
[0097] The standardization preprocessed myoelectric signal is subjected to semantic level segmentation processing to generate a segmentation result;
[0098] The segmentation result and the transfer result are optimized to remove part of the counterfactual and abnormal detection result, integrated and output to obtain a final predicted action sequence.
[0099] In addition, the logical instruction in the memory 730 described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the embodiments of the application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.
[0100] In another aspect, the present application also provides a computer program product comprising a computer program, which can be stored on a non-transitory computer readable storage medium, and the computer program can be executed by a processor to enable a computer to perform the above-mentioned method for continuous operation sequence of myoelectric signal segmentation, which comprises: obtaining myoelectric signals, and performing standardization preprocessing on the myoelectric signals;
[0101] performing semantic level segmentation processing on the myoelectric signals after standardization preprocessing to generate segmentation results;
[0102] recognizing and detecting the transition relationship between actions for the myoelectric signals after standardization preprocessing to generate transition results;
[0103] optimizing the segmentation results and the transition results, removing part of the counterfactual and abnormal detection results, integrating and outputting to obtain the final predicted action sequence.
[0104] In another aspect, the present application also provides a non-transitory computer readable storage medium having a computer program stored thereon, and the computer program can be executed by a processor to implement the above-mentioned method for continuous operation sequence of myoelectric signal segmentation, which comprises: obtaining myoelectric signals, and performing standardization preprocessing on the myoelectric signals;
[0105] performing semantic level segmentation processing on the myoelectric signals after standardization preprocessing to generate segmentation results;
[0106] recognizing and detecting the transition relationship between actions for the myoelectric signals after standardization preprocessing to generate transition results;
[0107] optimizing the segmentation results and the transition results, removing part of the counterfactual and abnormal detection results, integrating and outputting to obtain the final predicted action sequence.
[0108] The device embodiments described above are only schematic, wherein the units shown as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., they can be located in one place, or distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment. Those skilled in the art can understand and implement without creative labor.
[0109] Those skilled in the art can clearly understand the technical solutions of the various embodiments from the above description of the embodiments, and the various embodiments can be implemented by means of software with the necessary general hardware platforms, and of course, can also be implemented by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part of the prior art that makes a contribution, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, and the like, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0110] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features therein; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A method for segmenting electromyographic signals for continuous operation sequences, characterized in that, include: Acquire electromyographic signals and perform standardized preprocessing on the electromyographic signals; The standardized preprocessed electromyographic signals are subjected to semantic-level segmentation to generate segmentation results. The standardized preprocessed electromyographic signals are used to identify and detect the transfer relationships between movements, and to generate transfer results. The segmentation and transfer results are optimized, some counterfactual and anomaly detection results are removed, and the results are integrated and output to obtain the final predicted action sequence. The step of performing semantic-level segmentation on the standardized preprocessed electromyographic signals to generate segmentation results specifically includes: The standardized preprocessed electromyographic signals are input into a preset semantic segmentation model, and the operation type corresponding to each time point is output after processing. The electromyographic signals during the operation are classified at the time resolution level using a U-net-like structure to form a segmentation result. The process of identifying and detecting the transfer relationships between actions from standardized preprocessed electromyographic signals and generating transfer results specifically includes: The standardized preprocessed electromyographic signal is input into a preset transfer time detection model, and the model outputs whether a change in movement has occurred at the current time point. The switching time between actions is detected and identified using a U-net-like structure consistent with the semantic segmentation module, and the transition result is generated. The U-net-like structure is to change the two 2D convolution layers used in the original U-net to a 1D convolution and a depthwise separable convolution during the encoding process.
2. The electromyography signal segmentation method for continuous operation sequences according to claim 1, characterized in that, The acquisition of electromyographic signals, and the standardization preprocessing of the electromyographic signals, specifically include: The raw electromyographic (EMG) signals are acquired, and then preprocessed by decentering and standardization. The processing steps are as follows: Where E represents electromyography signal, ave represents mean calculation, and std represents standard deviation calculation.
3. The electromyography signal segmentation method for continuous operation sequences according to claim 1, characterized in that, The segmentation and transfer results are optimized by removing some counterfactual and anomaly detection results, specifically including: The segmentation and transfer results are input into a preset action sequence optimization model. After optimization by the action sequence optimization model, the sequence action group is output. The sequence of actions includes the type of action and the start time, and some counterfactual and abnormal detection results are removed.
4. The electromyography signal segmentation method for continuous operation sequences according to claim 1, characterized in that, The process of integrating and outputting the results to obtain the final predicted action sequence specifically includes: The action with the highest probability of occurrence of the transition time interval between adjacent actions in the detection results after removing some counterfactual and anomaly detections is taken as the action in the current time period; By integrating actions from different time periods, the final predicted action sequence is obtained.
5. An electromyography signal segmentation system for continuous operation sequences, characterized in that, The system includes: The data preprocessing module is used to acquire electromyographic signals and perform standardized preprocessing on the electromyographic signals; The semantic segmentation module is used to perform semantic-level segmentation on the standardized preprocessed electromyographic signals and generate segmentation results. The transfer time detection module is used to identify and detect the transfer relationship between actions from the standardized preprocessed electromyographic signals and generate transfer results; The action sequence optimization module is used to optimize the segmentation and transfer results, and remove some counterfactual and abnormal detection results; The results output module is used to integrate the outputs and obtain the final predicted action sequence. The step of performing semantic-level segmentation on the standardized preprocessed electromyographic signals to generate segmentation results specifically includes: The standardized preprocessed electromyographic signals are input into a preset semantic segmentation model, and the operation type corresponding to each time point is output after processing. The electromyographic signals during the operation are classified at the time resolution level using a U-net-like structure to form a segmentation result. The process of identifying and detecting the transfer relationships between actions from standardized preprocessed electromyographic signals and generating transfer results specifically includes: The standardized preprocessed electromyographic signal is input into a preset transfer time detection model, and the model outputs whether a change in movement has occurred at the current time point. The switching time between actions is detected and identified using a U-net-like structure consistent with the semantic segmentation module, and the transition result is generated. The U-net-like structure is to change the two 2D convolution layers used in the original U-net to a 1D convolution and a depthwise separable convolution during the encoding process.
6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the electromyographic signal segmentation method for continuous operation sequences as described in any one of claims 1 to 4.
7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the electromyographic signal segmentation method for continuous operation sequences as described in any one of claims 1 to 4.
8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the electromyographic signal segmentation method for continuous operation sequences as described in any one of claims 1 to 4.