Human muscle state prediction method, device and equipment

Abstract

The invention provides a human body muscle state prediction method, device and equipment, and relates to the technical field of muscle state prediction, and the method comprises the steps: obtaining target human body posture data of a target object through a motion capture technology; inputting the target human body posture data into a posture-muscle state mapping model to obtain a predicted muscle state change sequence corresponding to the target object output by the posture-muscle state mapping model; the posture-muscle state mapping model is obtained by training on the basis of the electromyographic signal sample data and posture sample data corresponding to the electromyographic signal sample data. According to the technical scheme, the muscle state is predicted through the human body posture data and the pre-trained posture-muscle state mapping model, and compared with the prior art, electromyographic signals do not need to be collected in real time, so that the human and material resource cost is reduced, and the use scene and the application environment are expanded.

Description

Technical Field

[0001] The present invention relates to the technical field of muscle state prediction, and in particular to a method, device and equipment for predicting human muscle state. Background Art

[0002] The assessment of human muscle status is widely used in intelligent perception scenarios such as rehabilitation assessment, motion analysis, human-computer interaction, and movement quality assessment.

[0003] Existing technologies primarily assess muscle status through real-time acquisition of electromyographic signals. This requires the subject to continuously wear electrodes, resulting in high costs. Furthermore, electromyographic signals are sensitive to factors such as electrode fit and environmental interference. They are susceptible to noise in non-laboratory environments, resulting in reduced signal quality. This significantly limits the environment in which they can be used, making widespread application difficult in both time and space. Summary of the Invention

[0004] The present invention provides a method, device and equipment for predicting human muscle state, which is used to solve the defects of the existing technology that the real-time acquisition of electromyographic signals is costly, has significant restrictions on the use environment, and is not convenient for wide application in time and space. The technical solution of the present invention predicts the muscle state through human posture data and a pre-trained posture-muscle state mapping model. Compared with the existing technology, there is no need to collect electromyographic signals in real time, which reduces human labor costs and expands the use scenarios and applicable environments.

[0005] The present invention provides a method for predicting human muscle state, comprising the following steps.

[0006] Acquire target human body posture data of the target object through motion capture technology; The target human body posture data is input into a posture-muscle state mapping model to obtain a predicted muscle state change sequence corresponding to the target object output by the posture-muscle state mapping model; the posture-muscle state mapping model is trained based on electromyographic signal sample data and posture sample data corresponding to the electromyographic signal sample data.

[0007] According to a human muscle state prediction method provided by the present invention, the posture-muscle state mapping model is obtained by training through the following steps: Acquiring the electromyographic signal sample data and the posture sample data; The electromyographic signal sample data and the posture sample data are respectively segmented based on a preset action cycle to obtain target segment electromyographic signals and target segment posture data corresponding to a plurality of action cycles; Supervise and iteratively train an untrained posture - muscle state mapping model based on the target paragraph EMG signals and target paragraph posture data corresponding to all the action cycles to obtain the posture - muscle state mapping model.

[0008] According to a human muscle state prediction method provided by the present invention, the step of respectively segmenting the EMG signal sample data and the posture sample data based on a preset action cycle to obtain target paragraph EMG signals and target paragraph posture data corresponding to each action cycle includes: Respectively segment the EMG signal sample data and the posture sample data based on a preset action cycle to obtain paragraph EMG signals and paragraph posture data corresponding to each action cycle; For each action cycle, perform noise reduction processing on the paragraph EMG signal corresponding to the action cycle to obtain the target paragraph EMG signal corresponding to the action cycle; Perform interpolation and completion processing on the paragraph posture data corresponding to the action cycle to obtain the target paragraph posture data corresponding to the action cycle.

[0009] According to a human muscle state prediction method provided by the present invention, the step of supervising and iteratively training an untrained posture - muscle state mapping model based on the target paragraph EMG signals and target paragraph posture data corresponding to all the action cycles to obtain the posture - muscle state mapping model includes: For each action cycle, perform sliding window segmentation on the target paragraph EMG signal data corresponding to the action cycle based on a preset unit window time to obtain a plurality of window EMG signals corresponding to the target paragraph EMG signal; For each window EMG signal, determine the manual features of the window EMG signal based on the window EMG signal; Cluster each window EMG signal based on the manual features corresponding to all the window EMG signals to obtain an EMG symbol sequence corresponding to the action cycle; Input the EMG symbol sequences and target paragraph posture data corresponding to all the action cycles into the untrained posture - muscle state mapping model for supervised learning iterative training to obtain the posture - muscle state mapping model.

[0010] According to a human muscle state prediction method provided by the present invention, the manual features include one or more of root mean square value, zero crossing rate, slope change, waveform length, mean absolute value, Willison amplitude, autoregressive coefficient, mean frequency, median frequency, and power spectrum ratio.

[0011] A method for predicting human muscle state provided by the present invention, clustering each of the window EMG signals based on the manual features corresponding to all the window EMG signals to obtain the EMG symbol sequence corresponding to the action period, includes: Using the K-means clustering algorithm, clustering each of the window EMG signals based on the manual features corresponding to all the window EMG signals to obtain the cluster category numbers respectively corresponding to each of the window EMG signals; Determining the EMG symbol sequence corresponding to the action period based on the cluster category numbers respectively corresponding to all the window EMG signals.

[0012] A method for predicting human muscle state provided by the present invention, the untrained pose-muscle state mapping model is specifically used for: In each supervised learning iteration round, for each of the action periods, sliding window segmentation is performed on the target paragraph pose data corresponding to the action period based on the preset unit window time to obtain a plurality of window pose data corresponding to the target paragraph pose data; For each of the window pose data, feature extraction is performed on the window pose data to obtain the window pose features corresponding to the window pose data; Performing supervised learning iterative training based on all the window pose features and all the EMG symbol sequences corresponding to the action period.

[0013] A method for predicting human muscle state provided by the present invention, the supervised learning iterative training based on all the window pose features and all the EMG symbol sequences corresponding to the action period, includes: For each of the window pose features, using the time attention mechanism to weight the key time periods of the window pose features in the time dimension to obtain the window time pose features corresponding to the window pose features; Using the spatial attention mechanism to weight the key limb regions of the window pose features in the spatial dimension to obtain the window space pose features corresponding to the window pose features; Performing spatio-temporal fusion processing on all the window time pose features and all the window space pose features to obtain the pose representation vector corresponding to the action period; Performing the supervised learning iterative training based on the pose representation vector and the EMG symbol sequence with time alignment.

[0014] The present invention also provides a device for predicting human muscle state, including the following modules: An acquisition module, configured to acquire the target human body pose data of the target object through an action capture technology; A prediction module, configured to input the target human body posture data into a posture-muscle state mapping model, and obtain a predicted muscle state change sequence corresponding to the target object output by the posture-muscle state mapping model; the posture-muscle state mapping model is trained based on electromyogram signal sample data and posture sample data corresponding to the electromyogram signal sample data.

[0015] The present invention further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the human muscle state prediction method as described in any one of the above is implemented.

[0016] The present invention further provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the human muscle state prediction method as described in any one of the above is implemented.

[0017] The present invention further provides a computer program product, including a computer program. When the computer program is executed by a processor, the human muscle state prediction method as described in any one of the above is implemented.

[0018] The human muscle state prediction method, device, and equipment provided by the present invention obtain target human body posture data of a target object through a motion capture technology; input the target human body posture data into a posture-muscle state mapping model, and obtain a predicted muscle state change sequence corresponding to the target object output by the posture-muscle state mapping model; the posture-muscle state mapping model is trained based on electromyogram signal sample data and posture sample data corresponding to the electromyogram signal sample data. The technical solution of the present invention predicts the muscle state through human body posture data and a pre-trained posture-muscle state mapping model, and compared with the prior art, it does not need to collect electromyogram signals in real time, reduces the labor and material costs, and expands the usage scenarios and applicable environments. Description of the Drawings

[0019] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0020] Figure 1 It is a schematic flowchart of the human muscle state prediction method provided by the present invention.

[0021] Figure 2 It is a schematic flowchart of the training of the posture-muscle state mapping model provided by the present invention.

[0022] Figure 3It is a schematic structural diagram of the human muscle state prediction device provided by the present invention.

[0023] Figure 4 It is a schematic structural diagram of the electronic device provided by the present invention. Detailed implementation manners

[0024] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without making creative efforts fall within the protection scope of the present invention.

[0025] In view of the above problems in the prior art, the present invention provides a method for predicting human muscle state, Figure 1 It is one of the schematic flowcharts of the human muscle state prediction method provided by the present invention. As Figure 1 shown, the method includes the following steps 110 to step 120.

[0026] Step 110: Obtain the target human body posture data of the target object through motion capture technology.

[0027] Specifically, first, the target object whose muscle state is to be detected can be determined. After determining the target object, the target human body posture data of the target object can be obtained through motion capture technology. Among them, the target human body posture data is the time series data representing the change of the target object's human body posture. Motion capture technology is a technology that records the motion information of an object or a human body and converts it into digital data. The motion capture technology can include one or more of mechanical motion capture (such as using wearable sensors), electromagnetic motion capture, optical motion capture (such as using a camera to capture the motion image of the target object), and inertial motion capture. The embodiments of the present invention do not make specific limitations here.

[0028] Step 120: Input the target human body posture data into the posture-muscle state mapping model to obtain the predicted muscle state change sequence corresponding to the target object output by the posture-muscle state mapping model; the posture-muscle state mapping model is trained based on the electromyogram signal sample data and the posture sample data corresponding to the electromyogram signal sample data.

[0029] Specifically, the electromyogram signal sample data can be obtained in advance. After obtaining the electromyogram signal sample data and the posture sample data, the untrained posture-muscle state mapping model can be trained based on the electromyogram signal sample data and the posture sample data to obtain the trained posture-muscle state mapping model.

[0030] Furthermore, the target human body posture data can be input into a posture-muscle state mapping model. The posture-muscle state mapping model can predict the change in muscle state during the corresponding action according to the mapping relationship between the human body posture data and the muscle state. Furthermore, a predicted muscle state change sequence corresponding to the target object output by the posture-muscle state mapping model can be obtained. The predicted muscle state change sequence characterizes the change in muscle state of the target object during the execution of the action corresponding to the target human body posture data.

[0031] The human muscle state prediction method provided by the present invention obtains the target human body posture data of the target object through motion capture technology; inputs the target human body posture data into a posture-muscle state mapping model to obtain a predicted muscle state change sequence corresponding to the target object output by the posture-muscle state mapping model; the posture-muscle state mapping model is trained based on electromyogram signal sample data and posture sample data corresponding to the electromyogram signal sample data. The technical solution of the present invention predicts the muscle state through the human body posture data and the pre-trained posture-muscle state mapping model. Compared with the prior art, it does not need to collect electromyogram signals in real time, reduces the labor and material costs, and expands the usage scenarios and applicable environments.

[0032] In one embodiment, the posture-muscle state mapping model is trained through the following steps: Obtain the electromyogram signal sample data and the posture sample data; Based on a preset action period, the electromyogram signal sample data and the posture sample data are respectively segmented to obtain target paragraph electromyogram signals and target paragraph posture data corresponding to each of the multiple action periods; Based on the target paragraph electromyogram signals and target paragraph posture data corresponding to all the action periods, a non-trained posture-muscle state mapping model is subjected to supervised learning iterative training to obtain the posture-muscle state mapping model.

[0033] Specifically, Figure 2 is a schematic flowchart of the training of the posture-muscle state mapping model provided by the present invention. As Figure 2 shown, the electromyogram signal sample data can include electromyogram signals corresponding to multiple actions of multiple objects. It is easy to understand that the posture data in the posture sample data corresponds to the electromyogram signals in the electromyogram signal sample data for the same action of the same object. The richer the objects and actions corresponding to the electromyogram signal sample data and the posture sample data, the better the effect of the subsequently trained posture-muscle state mapping model may be.

[0034] After obtaining the electromyogram signal sample data and the pose sample data, the electromyogram signal sample data and the pose sample data can be segmented respectively based on a preset action period, so as to realize the extraction of a single action segment of the electromyogram signal sample data and the pose sample data, and obtain the target segment electromyogram signal and the target segment pose data corresponding to each of the multiple action periods. Among them, the preset action period may include a pre-annotated action start point and an action end point.

[0035] Furthermore, the un-trained pose-muscle state mapping model can be iteratively trained through supervised learning based on the target segment electromyogram signals and the target segment pose data corresponding to all action periods, so as to obtain the pose-muscle state mapping model.

[0036] In the above embodiment, the electromyogram signal sample data and the pose sample data are segmented through a preset action period, which facilitates subsequent feature extraction and segmented modeling for the data corresponding to each action period respectively.

[0037] In one embodiment, the step of segmenting the electromyogram signal sample data and the pose sample data respectively based on a preset action period to obtain the target segment electromyogram signals and the target segment pose data corresponding to each of the multiple action periods includes: Segmenting the electromyogram signal sample data and the pose sample data respectively based on a preset action period to obtain the segment electromyogram signals and the segment pose data corresponding to each of the action periods; For each of the action periods, performing noise reduction processing on the segment electromyogram signal corresponding to the action period to obtain the target segment electromyogram signal corresponding to the action period; Performing interpolation and completion processing on the segment pose data corresponding to the action period to obtain the target segment pose data corresponding to the action period.

[0038] Specifically, as Figure 2 shown, the electromyogram signal sample data and the pose sample data can be segmented respectively based on a preset action period, and the segment electromyogram signals and the segment pose data corresponding to each of the action periods can be obtained.

[0039] Due to the problems of low-frequency baseline drift and high-frequency electrical interference commonly existing in the original electromyogram signals, noise reduction processing can be performed on the segment electromyogram signals to suppress the interference of non-physiological signal components. For each action period, noise reduction processing can be performed on the segment electromyogram signal corresponding to the action period. For example, a band-pass filter with a frequency range of 20 Hz to 450 Hz can be introduced to perform noise reduction processing on the segment electromyogram signal, so as to obtain the target segment electromyogram signal corresponding to the action period.

[0040] Since the original pose data often lacks some key-point frame data due to occlusion and loss of reflective markers. To ensure the continuity of the key-point (such a key-point can be, for example, the elbow point of the upper limb, etc., which represents the key-points of human actions) trajectory and the integrity of the spatial structure, a time interpolation completion strategy can be adopted. By referring to the time trend of adjacent frames and the spatial trajectory of the same group of key-points, the position of the key-points in the missing frames is interpolated and repaired. That is, for each action cycle, the pose data of the corresponding paragraph of the action cycle can be interpolated and completed to obtain the target paragraph pose data corresponding to the action cycle.

[0041] In the above embodiment, the paragraph EMG signals are denoised to suppress the interference of non-physiological signal components, and the paragraph pose data is also interpolated and completed to ensure the continuity of the key-point trajectory. The obtained target paragraph EMG signal data and target paragraph pose data have higher quality and stronger consistency, laying a foundation for subsequent model training.

[0042] In one embodiment, the unsupervised pose-muscle state mapping model is iteratively trained by supervised learning based on the target paragraph EMG signals and target paragraph pose data corresponding to all the action cycles to obtain the pose-muscle state mapping model, including: For each of the action cycles, the target paragraph EMG signal data corresponding to the action cycle is segmented by a sliding window based on a preset unit window time to obtain a plurality of window EMG signals corresponding to the target paragraph EMG signal; For each of the window EMG signals, the manual features of the window EMG signal are determined based on the window EMG signal; Based on the manual features corresponding to all the window EMG signals, each of the window EMG signals is clustered to obtain the EMG symbol sequence corresponding to the action cycle; The EMG symbol sequences corresponding to all the action cycles and the target paragraph pose data are input into the unsupervised pose-muscle state mapping model for iterative training by supervised learning to obtain the pose-muscle state mapping model.

[0043] Specifically, EMG signals are high-dimensional, continuous and non-stationary time series, with strong individual differences and muscle group specificities. Their data patterns are complex and it is difficult to directly establish a mapping relationship with pose data. To solve this problem, it is necessary to convert EMG signals into discrete, stable and interpretable EMG symbol sequences, which can retain key information while effectively compressing. Therefore, as Figure 2As shown, for each action cycle, first, the target paragraph EMG signal data corresponding to the action cycle can be segmented by sliding window based on a preset unit window time, and a plurality of window EMG signals corresponding to the target paragraph EMG signal are obtained. Among them, the overlapping rate corresponding to the sliding window segmentation can be set as needed. For example, it can be 50%. The preset unit window time can also be set as needed. The embodiments of the present invention do not make specific limitations here. For example, if the preset unit window time is 50 milliseconds, the time period corresponding to a single window EMG signal is also 50 milliseconds.

[0044] Further, for each window EMG signal, the manual features of the window EMG signal can be determined based on the window EMG signal. It should be noted that the manual features can include one or more features. For example, it can include classic manual features such as root mean square value, zero crossing rate, and slope change. Furthermore, based on the manual features corresponding to all window EMG signals, each window EMG signal can be clustered to obtain the EMG symbol sequence corresponding to the action cycle.

[0045] After obtaining the EMG symbol sequences corresponding to each action cycle respectively, the EMG symbol sequences corresponding to all action cycles and the target paragraph pose data can be input into an untrained pose-muscle state mapping model for supervised learning iterative training to obtain a trained pose-muscle state mapping model.

[0046] In the above embodiment, the EMG signal is segmented by sliding window, which ensures that the subsequent EMG signal can be transformed into a discrete, stable, and interpretable EMG symbol sequence, so that when training the pose-muscle state mapping model, the mapping relationship between the EMG symbol sequence and the target paragraph pose data can be established.

[0047] In one embodiment, the manual features include one or more of root mean square value, zero crossing rate, slope change, waveform length, mean absolute value, Willison amplitude, autoregressive coefficient, mean frequency, median frequency, and power spectrum ratio.

[0048] Specifically, the manual features include one or more of the following classical manual features: (1) Root mean square: used to measure the energy level of the electrical signal and reflect the muscle contraction intensity; (2) Zero crossing rate: used to measure the number of times the electromyogram signal passes through zero and reflect the fluctuation frequency of the electromyogram signal; (3) Slope change: used to count the number of times the sign of the slope of the electromyogram signal changes and reflect the change in the waveform direction of the electromyogram signal; (4) Waveform length: used to measure the complexity of the electromyogram signal waveform and reflect the "undulation" degree of muscle activity in the electromyogram signal; (5) Average absolute value: represents the average value of the absolute values of the electromyogram signal and reflects the overall amplitude of the electromyogram signal; (6) Willison amplitude: used to evaluate the frequency of amplitude changes in the electromyogram signal, count the number of amplitude changes in the electromyogram signal that exceed a preset threshold, and the preset threshold can be set as needed; (7) Autoregressive coefficient: describes the time series characteristics of the electromyogram signal through an autoregressive model and reflects the linear relationship between the electromyogram signal and its past values; (8) Average frequency: represents the weighted average of all frequency components in the electromyogram signal spectrum; (9) Median frequency: represents the frequency point corresponding to half of the total power in the power spectrum of the electromyogram signal; (10) Power spectrum ratio: represents the ratio of the power of the high-frequency band to the power of the low-frequency band in the electromyogram signal spectrum and is used to evaluate the fatigue state of the muscle. It should be noted that the manual features may also include other manual features other than the above features, and the embodiments of the present invention do not make specific limitations.

[0049] In the above embodiment, one or more features among the root mean square value, zero crossing rate, slope change, waveform length, average absolute value, Willison amplitude, autoregressive coefficient, average frequency, median frequency, and power spectrum ratio can take into account both time-domain and frequency-domain information and comprehensively reflect the amplitude, frequency, and dynamic characteristics of muscle contraction.

[0050] In one embodiment, clustering the window electromyogram signals based on the manual features corresponding to all the window electromyogram signals to obtain the electromyogram symbol sequence corresponding to the action period includes: Using the K-means clustering algorithm, clustering the window electromyogram signals based on the manual features corresponding to all the window electromyogram signals to obtain the cluster category numbers corresponding to the window electromyogram signals respectively; Determining the electromyogram symbol sequence corresponding to the action period based on the cluster category numbers corresponding to all the window electromyogram signals respectively.

[0051] Specifically, the K-means clustering algorithm is an unsupervised learning algorithm. Its goal is to divide the data set into K clusters, and each cluster is represented by the mean (i.e., the cluster center) closest to it. The "unsupervised" here means that the algorithm does not require pre-labeled training data to guide the clustering process during execution and completely discovers the internal structure of the data based on the characteristics of the data itself.

[0052] In this embodiment, the K-means clustering algorithm can be adopted to cluster each window electromyogram signal based on the manual features corresponding to all window electromyogram signals. The objective function corresponding to the K-means clustering algorithm can be expressed by the following formula: Wherein, represents minimization, represents the th cluster, represents the number of clusters, represents the cluster center of the th cluster, represents the Euclidean distance, represents the window electromyogram signal corresponding manual features.

[0053] In the above K-means clustering algorithm, in order to determine a reasonable number of clusters , the elbow method (ElbowMethod) can be adopted, that is, the sum of squared errors (SSE) under different values can be calculated, and the SSE change curve can be plotted. When gradually increases, the SSE will decrease rapidly, but after a certain point, the decline rate of the SSE will slow down significantly. The corresponding to this inflection point is the optimal number of clusters.

[0054] After clustering each window electromyogram signal, the cluster category numbers corresponding to each window electromyogram signal can be obtained. For example, the cluster category numbers corresponding to all window electromyogram signals can include 0, 1, 2... K -1. This cluster category number is the electromyogram symbol corresponding to the window electromyogram signal. Furthermore, the electromyogram symbol sequence corresponding to this action cycle can be determined based on the cluster category numbers corresponding to all window electromyogram signals respectively. It is easy to understand that this electromyogram symbol sequence can characterize the muscle state corresponding to this action cycle. Thus, the continuous signal of each muscle group is converted into a discrete symbol sequence, forming a temporal trajectory of muscle activation.

[0055] In the above embodiment, the symbolization of electromyogram signals is realized through the K-means clustering algorithm. This symbolization process constructs a stable and interpretable intermediate representation, which is convenient for subsequent semantic fusion with pose data. And it can significantly reduce the input dimension while retaining the discriminative state information in muscle activities, improving the stability and migration efficiency of subsequent models.

[0056] In one embodiment, the untrained pose-muscle state mapping model is specifically used for: In each supervised learning iteration, for each of the action cycles, based on the preset unit window time, perform sliding window segmentation on the target paragraph pose data corresponding to the action cycle to obtain a plurality of window pose data corresponding to the target paragraph pose data; For each of the window pose data, perform feature extraction on the window pose data to obtain window pose features corresponding to the window pose data; Perform supervised learning iterative training based on all the window pose features and all the electromyogram symbol sequences corresponding to the action cycle.

[0057] Specifically, since pose data is usually collected through motion capture and contains three-dimensional key point coordinates of multiple points, while electromyogram signals are muscle electrical activity signals that vary with time steps, in each supervised learning iteration, in order to ensure the consistency of pose data and electromyogram signals in the time dimension, for each action cycle, it is necessary to perform sliding window segmentation on the target paragraph pose data corresponding to the action cycle based on the above-mentioned preset unit window time to obtain a plurality of window pose data corresponding to the target paragraph pose data. It is easy to understand that the number of window pose data corresponding to the same action cycle is the same as the number of window electromyogram signals.

[0058] Furthermore, for each window pose data, feature extraction can be performed on the window pose data to obtain window pose features corresponding to the window pose data. Then, supervised learning iterative training can be performed based on all the window pose features and all the electromyogram symbol sequences corresponding to the action cycle.

[0059] In the above embodiment, performing sliding window segmentation on the target paragraph pose data corresponding to the action cycle based on the preset unit window time ensures the time alignment of the pose data and the electromyogram signals. This enables the window pose data and the window electromyogram signals to correspond one by one, and subsequent fusion analysis can also effectively predict the muscle activity state based on data from the same time period.

[0060] In one embodiment, the supervised learning iterative training based on all the window pose features and all the electromyogram symbol sequences corresponding to the action cycle includes: For each of the window pose features, use a time attention mechanism in the time dimension to weight the key time periods of the window pose features to obtain window time pose features corresponding to the window pose features; Use a spatial attention mechanism in the spatial dimension to weight the key limb regions of the window pose features to obtain window spatial pose features corresponding to the window pose features; Perform spatio-temporal fusion processing on all the window time pose features and all the window spatial pose features to obtain a pose representation vector corresponding to the action cycle; Perform the supervised learning iterative training based on the pose representation vector and the EMG symbol sequence aligned in time.

[0061] Specifically, the temporal attention mechanism can focus on the temporal features of key pose changes within the action cycle. Within each time window and between windows, it is possible to weight and focus on key time periods that are important for action classification, ensuring that the pose-muscle state mapping model can effectively capture the key phases of the action. That is, for the pose features of each window, the temporal attention mechanism can be used in the time dimension to weight the key time periods of the pose features of that window, obtaining the window temporal pose features corresponding to the pose features of that window.

[0062] Pose data usually includes multiple key points, and the spatial distribution and mutual relationships of these key points are crucial for action recognition. The spatial attention mechanism can effectively analyze these spatial correlations, enhance the modeling ability of inter-joint coordination, and thus improve the prediction ability of pose data for EMG symbols. Through the spatial attention mechanism, the pose-muscle state mapping model can comprehensively analyze the pose changes and spatial relationships at each moment in the pose data, ensuring that the pose-muscle state mapping model not only considers changes in the time dimension but also can make full use of the spatial connections between key points. That is, for the pose features of each window, the spatial attention mechanism can be used in the spatial dimension to weight the key limb regions of the pose features of that window, obtaining the window spatial pose features corresponding to the pose features of that window. The key limb regions can, for example, include limb regions such as the left upper limb, right upper limb, torso, left lower limb, and right lower limb.

[0063] Furthermore, corresponding spatio-temporal fusion processing can be performed on all window temporal pose features and all window spatial pose features to obtain the pose representation vector corresponding to this action cycle. Furthermore, supervised learning iterative training can be performed based on the pose representation vector aligned in time and the EMG symbol sequence. During this supervised learning iterative training process, the EMG symbol sequence is the label corresponding to the pose data, that is, the supervision information. It should be noted that in step 120, the pose-muscle state mapping model converts the target human body pose data into a pose representation vector according to the above steps, and then determines the predicted muscle state change sequence corresponding to the target human body pose data according to the mapping relationship between the pose representation vector and the EMG symbol sequence.

[0064] In the above embodiments, the window pose features are weighted in the time dimension and the space dimension respectively. After further spatio-temporal fusion, the pose data at each moment can establish an effective mapping relationship with the corresponding myoelectric symbol label, further improving the prediction accuracy of the muscle activity state. At the same time, it is ensured that in scenarios where myoelectric signals cannot be collected in real time, such as in high-cost, space-constrained or equipment-inconvenient environments, the muscle state can still be accurately predicted based on the pose data, thereby making up for the information gap caused by the lack of myoelectric signals.

[0065] The human muscle state prediction device provided by the present invention will be described below. The human muscle state prediction device described below can be mutually corresponded and referred to with the human muscle state prediction method described above.

[0066] Figure 3 is a schematic structural diagram of the human muscle state prediction device provided by the present invention, as Figure 3 shown, the human muscle state prediction device 300 includes the following modules: An acquisition module 310, configured to acquire target human pose data of a target object through motion capture technology; A prediction module 320, configured to input the target human pose data into a pose-muscle state mapping model, and obtain a predicted muscle state change sequence corresponding to the target object output by the pose-muscle state mapping model; the pose-muscle state mapping model is trained based on myoelectric signal sample data and pose sample data corresponding to the myoelectric signal sample data.

[0067] The human muscle state prediction device provided by the present invention acquires target human pose data of a target object through motion capture technology; inputs the target human pose data into a pose-muscle state mapping model, and obtains a predicted muscle state change sequence corresponding to the target object output by the pose-muscle state mapping model; the pose-muscle state mapping model is trained based on myoelectric signal sample data and pose sample data corresponding to the myoelectric signal sample data. The technical solution of the present invention predicts the muscle state through human pose data and a pre-trained pose-muscle state mapping model, which reduces the human and material costs compared with the prior art without real-time collection of myoelectric signals, and expands the usage scenarios and applicable environments.

[0068] In one embodiment, the human muscle state prediction device further includes a model training module, and the model training module is specifically configured to: Acquire the myoelectric signal sample data and the pose sample data; Based on a preset action cycle, respectively segment the myoelectric signal sample data and the pose sample data to obtain target paragraph myoelectric signals and target paragraph pose data corresponding to each of the multiple action cycles; Supervise and iteratively train the untrained posture-muscle state mapping model based on the target paragraph EMG signals and target paragraph posture data corresponding to all the action cycles to obtain the posture-muscle state mapping model.

[0069] In one embodiment, the model training module is further specifically configured to: Based on a preset action cycle, segment the EMG signal sample data and the posture sample data respectively to obtain the paragraph EMG signals and paragraph posture data corresponding to each of the action cycles; For each of the action cycles, perform noise reduction processing on the paragraph EMG signal corresponding to the action cycle to obtain the target paragraph EMG signal corresponding to the action cycle; Perform interpolation and completion processing on the paragraph posture data corresponding to the action cycle to obtain the target paragraph posture data corresponding to the action cycle.

[0070] In one embodiment, the model training module is further specifically configured to: For each of the action cycles, perform sliding window segmentation on the target paragraph EMG signal data corresponding to the action cycle based on a preset unit window time to obtain a plurality of window EMG signals corresponding to the target paragraph EMG signal; For each of the window EMG signals, determine the manual features of the window EMG signal based on the window EMG signal; Cluster each of the window EMG signals based on the manual features corresponding to all the window EMG signals to obtain the EMG symbol sequence corresponding to the action cycle; Input the EMG symbol sequences corresponding to all the action cycles and the target paragraph posture data into the untrained posture-muscle state mapping model for supervised learning iterative training to obtain the posture-muscle state mapping model.

[0071] In one embodiment, the manual features include one or more of root mean square value, zero crossing rate, slope change, waveform length, average absolute value, Willison amplitude, autoregressive coefficient, average frequency, median frequency, and power spectrum ratio.

[0072] In one embodiment, the model training module is further specifically configured to: Adopt the K-means clustering algorithm to cluster each of the window EMG signals based on the manual features corresponding to all the window EMG signals to obtain the cluster category numbers corresponding to each of the window EMG signals; Determine the EMG symbol sequence corresponding to the action cycle based on the cluster category numbers corresponding to all the window EMG signals respectively.

[0073] In one embodiment, the untrained posture-muscle state mapping model is specifically configured to: In each supervised learning iteration, for each of the action cycles, based on the preset unit window time, perform sliding window segmentation on the target paragraph pose data corresponding to the action cycle to obtain a plurality of window pose data corresponding to the target paragraph pose data; For each of the window pose data, perform feature extraction on the window pose data to obtain window pose features corresponding to the window pose data; Perform supervised learning iterative training based on all the window pose features and all the electromyogram symbol sequences corresponding to the action cycle.

[0074] In one embodiment, the untrained pose - muscle state mapping model is further specifically configured to: For each of the window pose features, in the time dimension, use a time attention mechanism to weight the key time periods of the window pose features to obtain window time pose features corresponding to the window pose features; In the space dimension, use a space attention mechanism to weight the key limb regions of the window pose features to obtain window space pose features corresponding to the window pose features; Perform spatio - temporal fusion processing on all the window time pose features and all the window space pose features to obtain a pose representation vector corresponding to the action cycle; Perform the supervised learning iterative training based on the pose representation vector aligned in time and the electromyogram symbol sequence.

[0075] Figure 4 An example of the physical structure diagram of an electronic device is shown as Figure 4 shown. The electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440. Among them, the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call the logical instructions in the memory 430 to execute the human muscle state prediction method, and this method includes: Obtain the target human body pose data of the target object through an action capture technology; Input the target human body pose data into the pose - muscle state mapping model to obtain a predicted muscle state change sequence corresponding to the target object output by the pose - muscle state mapping model; the pose - muscle state mapping model is trained based on electromyogram signal sample data and pose sample data corresponding to the electromyogram signal sample data.

[0076] In addition, when the logical instructions in the above-mentioned memory 430 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0077] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the human muscle state prediction method provided by the above-mentioned various methods. The method includes: Obtaining target human body posture data of a target object through motion capture technology; Inputting the target human body posture data into a posture-muscle state mapping model to obtain a predicted muscle state change sequence corresponding to the target object output by the posture-muscle state mapping model; the posture-muscle state mapping model is trained based on electromyogram signal sample data and posture sample data corresponding to the electromyogram signal sample data.

[0078] On another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is implemented to execute the human muscle state prediction method provided by the above-mentioned various methods. The method includes: Obtaining target human body posture data of a target object through motion capture technology; Inputting the target human body posture data into a posture-muscle state mapping model to obtain a predicted muscle state change sequence corresponding to the target object output by the posture-muscle state mapping model; the posture-muscle state mapping model is trained based on electromyogram signal sample data and posture sample data corresponding to the electromyogram signal sample data.

[0079] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative work.

[0080] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0081] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or equivalently replace some of the technical features. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for predicting the state of human muscles, characterized in that, Including: Obtaining target human body pose data of a target object through motion capture technology; Inputting the target human body pose data into a pose-muscle state mapping model to obtain a predicted muscle state change sequence corresponding to the target object output by the pose-muscle state mapping model; the pose-muscle state mapping model is trained based on electromyogram signal sample data and pose sample data corresponding to the electromyogram signal sample data.

2. The method for predicting the human muscle state according to claim 1, wherein, The pose-muscle state mapping model is trained through the following steps: Obtaining the electromyogram signal sample data and the pose sample data; Based on a preset action cycle, respectively segmenting the electromyogram signal sample data and the pose sample data to obtain target paragraph electromyogram signals and target paragraph pose data corresponding to each action cycle; Based on the target paragraph electromyogram signals and target paragraph pose data corresponding to all the action cycles, performing supervised learning iterative training on an untrained pose-muscle state mapping model to obtain the pose-muscle state mapping model.

3. The human muscle state prediction method according to claim 2, characterized in that The step of based on a preset action cycle, respectively segmenting the electromyogram signal sample data and the pose sample data to obtain target paragraph electromyogram signals and target paragraph pose data corresponding to each action cycle includes: Based on a preset action cycle, respectively segmenting the electromyogram signal sample data and the pose sample data to obtain paragraph electromyogram signals and paragraph pose data corresponding to each action cycle; For each action cycle, performing noise reduction processing on the paragraph electromyogram signal corresponding to the action cycle to obtain the target paragraph electromyogram signal corresponding to the action cycle; Performing interpolation and completion processing on the paragraph pose data corresponding to the action cycle to obtain the target paragraph pose data corresponding to the action cycle.

4. The method for predicting the human muscle state according to claim 2, wherein, The step of based on the target paragraph electromyogram signals and target paragraph pose data corresponding to all the action cycles, performing supervised learning iterative training on an untrained pose-muscle state mapping model to obtain the pose-muscle state mapping model includes: For each action cycle, based on a preset unit window time, performing sliding window segmentation on the target paragraph electromyogram signal data corresponding to the action cycle to obtain a plurality of window electromyogram signals corresponding to the target paragraph electromyogram signal; For each window electromyogram signal, determining the manual features of the window electromyogram signal based on the window electromyogram signal; Clustering each window electromyogram signal based on the manual features corresponding to all the window electromyogram signals to obtain an electromyogram symbol sequence corresponding to the action cycle; Inputting the electromyogram symbol sequences corresponding to all the action cycles and the target paragraph pose data into the untrained pose-muscle state mapping model for supervised learning iterative training to obtain the pose-muscle state mapping model.

5. The human muscle state prediction method according to claim 4, characterized in that, The manual features include one or more of root mean square value, zero crossing rate, slope change, waveform length, average absolute value, Willison amplitude, autoregressive coefficient, average frequency, median frequency, and power spectrum ratio.

6. The method for predicting the human muscle state according to claim 4, wherein The step of clustering each window electromyogram signal based on the manual features corresponding to all the window electromyogram signals to obtain an electromyogram symbol sequence corresponding to the action cycle includes: Using the K-means clustering algorithm, cluster each of the window electromyogram signals based on the manual features corresponding to all the window electromyogram signals to obtain the cluster category numbers corresponding to each of the window electromyogram signals; Determine the electromyogram symbol sequence corresponding to the action cycle based on the cluster category numbers respectively corresponding to all the window electromyogram signals.

7. The method for predicting the human muscle state according to any one of claims 4 to 6, characterized in that The untrained posture-muscle state mapping model is specifically used for: In each supervised learning iteration round, for each of the action cycles, perform sliding window segmentation on the target passage posture data corresponding to the action cycle based on the preset unit window time to obtain a plurality of window posture data corresponding to the target passage posture data; For each of the window posture data, extract features from the window posture data to obtain the window posture features corresponding to the window posture data; Perform supervised learning iterative training based on all the window posture features and all the electromyogram symbol sequences corresponding to the action cycle.

8. The method for predicting the human muscle state according to claim 7, wherein The performing supervised learning iterative training based on all the window posture features and all the electromyogram symbol sequences corresponding to the action cycle includes: For each of the window posture features, perform weighted key time period on the window posture features in the time dimension using a time attention mechanism to obtain the window time posture features corresponding to the window posture features; Perform weighted key limb area on the window posture features in the space dimension using a space attention mechanism to obtain the window space posture features corresponding to the window posture features; Perform spatio-temporal fusion processing on all the window time posture features and all the window space posture features to obtain the posture representation vector corresponding to the action cycle; Perform the supervised learning iterative training based on the time-aligned posture representation vector and the electromyogram symbol sequence.

9. A human muscle state prediction device, characterized in that, including: An acquisition module, configured to acquire the target human body posture data of the target object through an action capture technology; A prediction module, configured to input the target human body posture data into the posture-muscle state mapping model to obtain the predicted muscle state change sequence corresponding to the target object output by the posture-muscle state mapping model; the posture-muscle state mapping model is trained based on electromyogram signal sample data and the posture sample data corresponding to the electromyogram signal sample data.

10. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the human muscle state prediction method according to any one of claims 1 to 8.

Images