Lower Limb Muscle Fatigue State Recognition Method and System Based on Multimodal Feature Fusion

Abstract

The present invention discloses a method and system for identifying the fatigue state of lower limb muscles based on multi-modal feature fusion. The method for identifying the fatigue state of lower limb muscles based on multi-modal feature fusion includes the following steps: Step S1, arranging multi-modal sensors on the object to be measured, and collecting signals through the multi-modal sensors; obtaining bioelectric signals, muscle morphology information, and force-position information through the multi-modal sensors; Step S2, obtaining the fatigue state through the fatigue state recognition model according to the bioelectric signals, muscle morphology information, and force-position information. The present invention discloses a method and system for identifying the fatigue state of lower limb muscles based on multi-modal feature fusion, which has the characteristics of high detection accuracy, fast dynamic response, and strong anti-interference ability.

Description

Technical Field

[0001] The present invention relates to the technical field of multimodal feature recognition, and particularly relates to a method and system for recognizing the fatigue state of lower limb muscles based on multimodal feature fusion. Background Art

[0002] Detecting muscle fatigue has far-reaching significance in the fields of sports science, rehabilitation medicine, occupational health, etc. Its real-time performance and accuracy are crucial for individual health management and sports performance optimization. Existing muscle fatigue detection methods have the following defects:

[0003] 1. Limitation of single data source: Existing technologies mostly rely on a single biological signal (such as surface electromyogram signal sEMG, heart rate, etc.), which is vulnerable to noise interference. High-frequency noise may cause feature loss or a single feature of joint angle, and cannot reflect the coupling relationship between the dynamic changes of muscle morphology and mechanical state;

[0004] 2. Insufficient analysis of muscle characteristics: Existing technologies lack the ability to capture real-time dynamics in the fatigue assessment of continuous repetitive movements, and do not consider the influence of muscle cross-section deformation and muscle deformation on fatigue. Bioelectric signal sensors are vulnerable to motion artifacts and environmental interference;

[0005] 3. Poor model generalization: The single feature dimension leads to insufficient classification accuracy of machine learning models for complex fatigue states. Summary of the Invention

[0006] The present invention overcomes the deficiencies of the prior art and provides a method and system for recognizing the fatigue state of lower limb muscles based on multimodal feature fusion, which has the characteristics of high detection accuracy, fast dynamic response, and strong anti-interference ability.

[0007] To achieve the above object, the technical solution adopted by the present invention is: A method for recognizing the fatigue state of lower limb muscles based on multimodal feature fusion, comprising the following steps:

[0008] Step S1, arranging multimodal sensors on the object to be measured, and collecting signals through the multimodal sensors; the signal collection includes: obtaining bioelectric signals, muscle morphology information, and force-position information through the multimodal sensors;

[0009] Step S2, obtaining the fatigue state through a fatigue state recognition model according to the bioelectric signals, muscle morphology information, and force-position information.

[0010] In a preferred embodiment of the present invention, the multimodal sensors include bioelectric signal sensors, muscle morphology collectors, and force-position information collection devices;

[0011] Collect the bioelectrical signals on the object to be measured through a bioelectrical signal sensor, collect the muscle morphology information on the object to be measured through a muscle morphology collector, and collect the force-position information on the object to be measured through a force-position information collection device.

[0012] In a preferred embodiment of the present invention, the myoelectric feature extraction is performed on the bioelectrical signals through a fatigue state recognition model, and the myoelectric feature extraction steps include:

[0013] Preprocess the bioelectrical signals collected by the multimodal sensor;

[0014] Then obtain the time-domain features, frequency-domain features, and non-linear features according to the preprocessed bioelectrical signals.

[0015] In a preferred embodiment of the present invention, the time-domain features include the root mean square (RMS), and the root mean square (RMS) includes:

[0016] , where X i is the voltage value of the i-th discrete sampling point in the myoelectric signal sequence; N is the total number of discrete data points within the myoelectric signal time window, and it is calculated for the surface electromyogram (sEMG) signal with a length of N;

[0017] And / or, the frequency-domain features include the median frequency (MF), and the median frequency (MF) includes: performing a Fourier transform on the preprocessed signal and calculating the power spectral density of the signal ;

[0018] , where ω is the signal frequency.

[0019] In a preferred embodiment of the present invention, the non-linear features include the sample entropy (SampEn), and the sample entropy (SampEn) includes:

[0020] For a time series x(n) = {x(1), x(2),..., x(N)} with a length of N, given the embedding dimension m, the original time series forms a vector group X m (i),

[0021] X m (i) = {x(i), x(i + 1),..., x(i + m - 1)}, where 1 ≤ i ≤ N - m + 1; define d[X m (i), X m (j)] as the maximum distance between the corresponding vectors in two different vector groups; n is the position index of the data points in the time series, representing the n-th time point;

[0022] d[X m (i), X m(j)] = max(|x(i + k) - x(j + k)|), where 0 ≤ k ≤ m - 1, 1 ≤ i, j ≤ N - m + 1, and i is not equal to j; given r, where r is the similarity tolerance; Defined as the ratio of the number of d[X m (i), X m (j)] ≤ r to the total number of vectors N - m + 1 is: ; where B i (r) is the number of d[X m (i), X m (j)] ≤ r; B i (r) = num{d[X m (i), X m (j)] ≤ r}, The average value of is denoted as , applicable to all i, ;

[0023] When the embedding dimension is m + 1, the ratio of the number of d[X m+1 (i), X m+1 (j)] ≤ r to the total number of vectors N - m is denoted as , , where A i (r) is the number of d[X m+1 (i), X m+1 (j)] ≤ r; A i (r) = num{d[X m+1 (i), X m+1 (j)] ≤ r}, and the average value of A i m (r) is denoted as D m+1 (r), applicable to all i, ; The sample entropy of the time series is:

[0024] .

[0025] In a preferred embodiment of the present invention, muscle morphology feature extraction is performed on muscle morphology information through a fatigue state recognition model. The muscle morphology feature extraction steps include:

[0026] According to the bioelectric signal sensor in the multi-modal sensor, the three-dimensional coordinates of the marked points set on the object to be measured are captured by an infrared optical motion capture system. Based on the three-dimensional coordinates of the marked points, the muscle cross-sectional circle radius, surface stretching length, and the change in the triangular area of the marked points are dynamically calculated.

[0027] In a preferred embodiment of the present invention, the calculation steps of the change in the triangular area of the marked points include: Based on Heron's formula, the area A of the triangle is calculated through the side lengths of the triangle;

[0028] ; , where a, b, and c are the three side lengths of a triangle;

[0029] The calculation steps of the radius of the muscle cross-sectional circle include: According to the calculation formula of the circumradius R calculate the circumradius, and the circumradius is the radius of the muscle cross-sectional circle;

[0030] And / or, the calculation steps of the surface stretching length include: Based on the radius of the muscle cross-sectional circle, calculate the central angle θ through the cosine theorem, and calculate the arc length according to the radius and the central angle, where the radius of the muscle cross-sectional circle: Central angle: , where R is the radius of the muscle cross-sectional circle, C is the chord length, and the arc length: , where arc is the arc length, and the surface stretching length is obtained by multiplying the radian value of the central angle by the radius due to the muscle surface strain.

[0031] In a preferred embodiment of the present invention, force-position joint features are extracted from the force-position information through a fatigue state recognition model, and the force-position joint feature extraction steps include:

[0032] Obtain the peak ground reaction force Fz-max by finding the maximum ground reaction force during the reverse jump, including the following steps: By traversing the maximum ground reaction force during the reverse jump, regard the maximum value as the peak ground reaction force Fz-max; , where is the function of the ground reaction force in the vertical direction Z changing with time, that is, the vertical component value of the ground reaction force at time t, is directly read by the force-position information acquisition device of the multi-modal sensor;

[0033] Calculate the reverse jump height through the vertical displacement of the hip joint marker point; including the following steps:

[0034] By traversing the highest point of the z-axis coordinate of the marker point during the reverse jump, subtract the z-axis coordinate at the resting state from the highest point value to obtain the reverse jump height , , where is the highest point of the z-axis coordinate of the marker point, is the z-axis coordinate at the resting state;

[0035] The calculation method of the joint angle includes the following steps:

[0036] An equivalent model of the object to be measured is established through three marked points arranged on the object to be measured, and the angles between relatively movable parts on the x, y, and z planes are calculated according to the equivalent model on the object to be measured, so as to obtain the joint angles of the object to be measured.

[0037] In a preferred embodiment of the present invention, the fatigue state recognition model includes a random forest algorithm, and the random forest algorithm includes: the basic unit uses a decision tree, and the Gini coefficient is used to select the optimal splitting feature and threshold;

[0038] The fatigue state recognition method of the fatigue state recognition model includes: dividing the fatigue level according to the joint angle offset and the force-position characteristics, and generating labels;

[0039] The performance of the model is evaluated through cross-validation, and the fatigue state is output.

[0040] In a preferred embodiment of the present invention, a lower limb muscle fatigue state recognition system based on multi-modal feature fusion includes: a fatigue state recognition model, and a multi-modal sensor that provides bioelectrical signals, muscle morphology information, and force-position information to the fatigue state recognition model;

[0041] The multi-modal sensor includes a bioelectrical signal sensor, a muscle morphology collector, and a force-position information acquisition device. The bioelectrical signal sensor, the muscle morphology collector, and the force-position information acquisition device are respectively interconnected with the fatigue state recognition model through a wireless electromyogram acquisition module, an infrared optical motion capture system, and a force platform system;

[0042] It is used to implement a method for recognizing the fatigue state of lower limb muscles based on multi-modal feature fusion.

[0043] The present invention solves the defects existing in the technical background, and the beneficial technical effects of the present invention are:

[0044] A method and system for recognizing the fatigue state of lower limb muscles based on multi-modal feature fusion have the characteristics of high detection accuracy, fast dynamic response, and strong anti-interference ability.

[0045] Multi-modal feature fusion further improves the fatigue recognition accuracy compared with the single electromyogram method.

[0046] The real-time synchronous acquisition of muscle cross-sectional morphology and mechanical parameters realizes rapid update of the fatigue state.

[0047] The introduction of multiple motion features reduces the interference of artifacts on the electromyogram signal. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The present invention will be further described below with reference to the drawings and embodiments.

[0049] Figure 1Schematic diagram of the arrangement of multi-modal sensors on the surface of the rectus femoris muscle in the preferred embodiment of the present invention;

[0050] Figure 2 Schematic diagram of the arrangement of multi-modal sensors at each joint during countermovement jumps in the preferred embodiment of the present invention Figure 1 (front);

[0051] Figure 3 Schematic diagram of the arrangement of multi-modal sensors at each joint during countermovement jumps in the preferred embodiment of the present invention Figure 2 (back);

[0052] Figure 4 Schematic diagram of the working process of the method and system for identifying the fatigue state of lower limb muscles based on multi-modal feature fusion in the preferred embodiment of the present invention;

[0053] Figure 5 Schematic diagram of the working process of using a random forest fatigue recognition model to identify the fatigue state in the method and system for identifying the fatigue state of lower limb muscles based on multi-modal feature fusion in the preferred embodiment of the present invention;

[0054] Figure 6 Curve graph of dynamic radius calculation in the method and system for identifying the fatigue state of lower limb muscles based on multi-modal feature fusion in the preferred embodiment of the present invention;

[0055] Figure 7 Schematic diagram of dynamic radius calculation in the method and system for identifying the fatigue state of lower limb muscles based on multi-modal feature fusion in the preferred embodiment of the present invention;

[0056] Figure 8 Schematic diagram of the calculation of electromyography features (taking root mean square as an example) in the method and system for identifying the fatigue state of lower limb muscles based on multi-modal feature fusion in the preferred embodiment of the present invention;

[0057] Figure 9 Schematic diagram of the calculation process of the muscle cross-sectional circle radius, arc length, and triangular area in the method and system for identifying the fatigue state of lower limb muscles based on multi-modal feature fusion in the preferred embodiment of the present invention;

[0058] Figure 10 Schematic diagram of obtaining the maximum ground reaction force in the method and system for identifying the fatigue state of lower limb muscles based on multi-modal feature fusion in the preferred embodiment of the present invention;

[0059] Figure 11 Schematic diagram of obtaining the maximum jump height in the method and system for identifying the fatigue state of lower limb muscles based on multi-modal feature fusion in the preferred embodiment of the present invention;

[0060] Figure 12Schematic diagram for obtaining the change amount of joint angle in the method and system for identifying the lower limb muscle fatigue state based on multi-modal feature fusion according to the preferred embodiment of the present invention;

[0061] Figure 13 Schematic diagram of the random forest algorithm process according to the preferred embodiment of the present invention. Detailed implementation manners

[0062] Now, the present invention will be further described in detail with reference to the accompanying drawings and embodiments. These drawings are all simplified schematic diagrams, only illustrating the basic structure of the present invention in a schematic manner, so they only show the components related to the present invention.

[0063] It should be noted that if there are directional indications (such as up, down, bottom, top, etc.) involved in the embodiments of the present invention, the directional indications are only used to explain the relative positional relationship and movement conditions between components in a certain specific posture. If the specific posture changes, the directional indications will also change accordingly. The terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. Unless otherwise clearly specified and limited, the terms "set", "connected", and "connected" should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral connection; it may be directly connected, or indirectly connected through an intermediate medium, and it may be the internal communication of two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances. Embodiment 1

[0064] As Figures 1-13 shown, a method for identifying the lower limb muscle fatigue state based on multi-modal feature fusion includes the following steps:

[0065] Step S1, arranging multi-modal sensors on the object to be measured, and collecting signals through the multi-modal sensors; the signal collection includes: obtaining bioelectrical signals, muscle morphology information, and force-position information through the multi-modal sensors;

[0066] Step S2, obtaining the fatigue state through the fatigue state recognition model according to the bioelectrical signals, muscle morphology information, and force-position information.

[0067] Specifically, the multi-modal sensors include bioelectrical signal sensors, muscle morphology collectors, and force-position information collection devices; the bioelectrical signals on the object to be measured are collected through the bioelectrical signal sensors, the muscle morphology information on the object to be measured is collected through the muscle morphology collectors, and the force-position information on the object to be measured is collected through the force-position information collection devices.

[0068] Specifically, an electromyogram feature extraction is performed on the bioelectric signal through a fatigue state recognition model. The electromyogram feature extraction steps include: preprocessing the bioelectric signal collected by the multi-modal sensor, and the preprocessing includes band-pass filtering, denoising, and segmentation; then obtaining time-domain features, frequency-domain features, and non-linear features according to the preprocessed bioelectric signal.

[0069] Specifically, a muscle morphology feature extraction is performed on the muscle morphology information through a fatigue state recognition model. The muscle morphology feature extraction steps include: capturing the three-dimensional coordinates of the markers set on the object to be measured by using an infrared optical motion capture system through the bioelectric signal sensor in the multi-modal sensor, and dynamically calculating the muscle cross-sectional circle radius, surface stretching length, and change in the triangular area of the markers according to the three-dimensional coordinates of the markers. Embodiment 2

[0070] As Figures 1-13 shown, a lower limb muscle fatigue state recognition method based on multi-modal feature fusion includes the following steps:

[0071] Step S1, arranging a multi-modal sensor on the object to be measured, and performing signal acquisition through the multi-modal sensor; the signal acquisition includes: obtaining bioelectric signals, muscle morphology information, and force-position information through the multi-modal sensor.

[0072] Step S2, obtaining the fatigue state through the fatigue state recognition model according to the bioelectric signal, muscle morphology information, and force-position information.

[0073] Specifically, the multi-modal sensor includes a bioelectric signal sensor, a muscle morphology collector, and a force-position information acquisition device; the bioelectric signal on the object to be measured is collected through the bioelectric signal sensor, the muscle morphology information on the object to be measured is collected through the muscle morphology collector, and the force-position information on the object to be measured is collected through the force-position information acquisition device.

[0074] Specifically, an electromyogram feature extraction is performed on the bioelectrical signal through a fatigue state recognition model. The electromyogram feature extraction steps include: preprocessing the bioelectrical signal collected by a multi-modal sensor, and the preprocessing includes band-pass filtering, denoising, and segmentation; then obtaining time-domain features, frequency-domain features, and non-linear features according to the preprocessed bioelectrical signal. More specifically, the preprocessing includes band-pass filtering, denoising, and segmentation processing. The preprocessing of the data is aimed at: the bioelectrical signal, muscle morphology information, and force-position information collected by a bioelectrical signal sensor (i.e., a wireless electromyogram acquisition module), a muscle morphology collector (i.e., an infrared optical motion capture system), and a force-position information acquisition device (i.e., a force platform system). Among them, the preprocessing includes band-pass filtering, denoising, and segmentation processing for the bioelectrical signal; and segmentation processing for the muscle morphology information and the force-position information. Among them, the band-pass filtering uses a Chebyshev type I filter (20 - 450 Hz); the denoising uses wavelet decomposition denoising, and the wavelet basis function is selected as Daubechies4 (db4), and the wavelet packet decomposition layer is 4.

[0075] Furthermore, the time-domain features include the root mean square RMS, and the root mean square RMS includes:

[0076] , where X i is the voltage value of the i-th discrete sampling point in the electromyogram signal sequence.

[0077] Furthermore, the frequency-domain features include the median frequency MF, and the median frequency MF includes: performing a Fourier transform on the preprocessed signal and calculating the power spectral density P(ω) of the signal, , where ω is the signal frequency.

[0078] Furthermore, the non-linear features include the sample entropy SampEn, and the sample entropy SampEn includes:

[0079] For a time series x(n) = {x(1), x(2),..., x(N)} of length N, given an embedding dimension m, the original time series forms a vector group X of length N - m + 1 m (i),

[0080] X m (i) = {x(i), x(i + 1),..., x(i + m - 1)}, where 1 ≤ i ≤ N - m + 1; define d[X m (i), X m (j)] as the maximum distance between the corresponding vectors in two different vector groups; n is the position index of the data points in the time series, representing the n-th time point;

[0081] d[X m (i), X m(j)] = max(|x(i + k) - x(j + k)|), where 0 ≤ k ≤ m - 1, 1 ≤ i, j ≤ N - m + 1, and i is not equal to j; given r, where r is the similarity tolerance; is defined as the ratio of the number of d[X m (i), X m (j)] ≤ r to the total number of vectors N - m + 1: ; where B i (r) is the number of d[X m (i), X m (j)] ≤ r; B i (r) = num{d[X m (i), X m (j)] ≤ r}, The average value of is denoted as , applicable to all i, ;

[0082] When the embedding dimension is m + 1, the ratio of the number of d[X m+1 (i), X m+1 (j)] ≤ r to the total number of vectors N - m is denoted as , , where A i (r) is the number of d[X m+1 (i), X m+1 (j)] ≤ r; A i (r) = num{d[X m+1 (i), X m+1 (j)] ≤ r}, A i m (r) The average value is denoted as D m+1 (r), applicable to all i, ; The sample entropy of the time series is:

[0083] .

[0084] Specifically, the muscle morphology features are extracted from the muscle morphology information through the fatigue state recognition model. The muscle morphology feature extraction steps include: capturing the three-dimensional coordinates of the set marker points on the object to be measured by the infrared optical motion capture system according to the bioelectric signal sensor in the multi-modal sensor, and dynamically calculating the muscle cross-sectional circle radius, surface stretching length, and change in the triangular area of the marker points based on the three-dimensional coordinates of the marker points.

[0085] Further, muscle morphological features are extracted from the muscle morphological information through a fatigue state recognition model. The steps of extracting the muscle morphological features include: capturing the three-dimensional coordinates of the set markers on the object to be measured by using an infrared optical motion capture system for the bioelectric signal sensor in the multimodal sensor, and dynamically calculating the muscle cross-sectional circle radius, surface stretching length, and change in the triangular area of the markers according to the three-dimensional coordinates of the markers.

[0086] Further, the calculation steps of the change in the triangular area of the markers include: based on Heron's formula, calculating the triangle area A through the side lengths of the triangle, ; , where a, b, and c are the three side lengths of the triangle.

[0087] Further, the calculation steps of the muscle cross-sectional circle radius include: calculating the circumradius according to the calculation formula of the circumradius R The circumradius is the muscle cross-sectional circle radius.

[0088] Further, the calculation steps of the surface stretching length include: based on the muscle cross-sectional circle radius, calculating the central angle θ through the cosine theorem, and calculating the arc length according to the radius and the central angle. Among them, the muscle cross-sectional circle radius: ; Central angle: , where R is the muscle cross-sectional circle radius, C is the chord length, and arc length: , where arc is the arc length, and the surface stretching length is obtained by multiplying the radian value of the central angle by the radius for the muscle surface strain.

[0089] Specifically, force-position joint features are extracted from the force-position information through a fatigue state recognition model. The steps of extracting the force-position joint features include:

[0090] Further, obtaining the peak ground reaction force Fz-max by finding the maximum ground reaction force during the countermovement jump includes the following steps: by traversing the maximum ground reaction force during the countermovement jump and regarding the maximum value as the peak ground reaction force Fz-max, , where, is the function of the ground reaction force in the vertical direction Z changing with time.

[0091] Further, calculating the countermovement jump height through the vertical displacement of the hip joint marker; includes the following steps:

[0092] By traversing the highest point of the marker z-axis coordinate during the countermovement jump and subtracting the z-axis coordinate at the resting state from the highest point value, the countermovement jump height is obtained , , where, is the highest point of the marker z-axis coordinate, is the z-axis coordinate at the resting state;

[0093] Further, the method for calculating the joint angle includes the following steps:

[0094] An equivalent model of the object to be measured is established through three marked points arranged on the object to be measured, and the angles between relatively movable parts on the x, y, and z planes are calculated according to the equivalent model on the object to be measured, so as to obtain the joint angle of the object to be measured. Embodiment III

[0095] On the basis of Embodiment II, the fatigue state recognition model includes a random forest algorithm, and the random forest algorithm includes: the basic unit adopts a decision tree, and the Gini coefficient is used to select the optimal splitting feature and threshold. Specifically, the number of decision trees is 300; the minimum number of leaf nodes is 10, and the maximum depth is 20. Embodiment IV

[0096] Based on Embodiment 3 or Embodiment 2, in this embodiment, the bioelectrical signal sensor (electromyogram signal acquisition device) uses an ErgoLAB surface electromyogram measurement system with a sampling frequency of 1000 Hz, including 8-channel acquisition electrodes, and supports Bluetooth 5.0 wireless signal transmission. The muscle morphology collector (motion capture system) uses a Vicon optical motion capture system, model Vicon Mx-GIGA NET, and its hardware includes 16 Vicon MX cameras, a PC host, a Vicon Datastation ADCPatchPanel information conversion box, and MX special connecting wires, etc., with a sampling frequency of 100 Hz. The force and position information acquisition device (force platform) uses a three-dimensional force platform in the prior art, which is built-in with six high-precision mechanical sensors, can collect the mechanical signals acting on the platform in real time, and can directly obtain the following mechanical parameters: Fx, Fy, Fz, Mx, My, Mz, three-dimensional force vector, center of pressure (COP), with a sampling frequency of 1000 Hz. More specifically, in this embodiment, the object to be measured is the lower limb. Therefore, in this embodiment, the electromyogram electrodes are arranged according to the anatomical positions of the main fatigued muscles of the lower limb (such as bilateral rectus femoris), and electromyogram electrodes are pasted on the surface of the muscle belly; the arrangement of motion capture markers (Mark points): Based on the muscle cross-section strain distribution law of the rectus femoris, reflective markers are evenly arranged horizontally on the muscle surface at equal intervals (spacing 40 mm), and at the same time, Mark points are pasted at the ankle, knee, and hip joints to synchronously capture the kinematic data of the countermovement jump; mechanical data acquisition: A pressure-sensitive area is set on the surface of the force platform to synchronously collect the three-dimensional ground reaction force during the countermovement jump. Among them, Fx: the force component along the X-axis direction, defined as the horizontal lateral force of the sensor (such as the front-back direction); Fy: the force component along the Y-axis direction, defined as the horizontal longitudinal force of the sensor (such as the left-right direction); Fz: the force component along the Z-axis direction, defined as the vertical force of the sensor (such as the up-down direction); Mx: the moment component around the X-axis, indicating the rotational effect of the object around the lateral axis of the sensor (such as torsion); My: the moment component around the Y-axis, indicating the rotational effect of the object around the longitudinal axis (such as tilt); Mz: the moment component around the Z-axis, indicating the rotational effect of the object around the vertical axis (such as rotation). Embodiment 5

[0097] Based on Embodiment 2, Embodiment 3, or Embodiment 4, the fatigue state recognition method of the fatigue state recognition model includes: dividing the fatigue level according to the joint angle offset and force-position characteristics, and generating labels.

[0098] Evaluate the model performance through cross-validation and output the fatigue state.

[0099] Specifically, the fatigue state recognition model includes a feature extraction module and a random forest algorithm including cross-validation; among them, the feature extraction module includes: electromyogram feature extraction, muscle morphology feature extraction, and force-position information extraction.

[0100] Random forest is an ensemble learning method that performs classification tasks by integrating multiple decision trees. For a dataset containing K samples, assuming each sample contains S features (the features are the above-mentioned electromyography features and muscle morphology features). Randomly draw K samples (with repetition allowed) from the dataset to form a training subset for a single decision tree. Repeat the sampling T times to generate T independent data subsets, corresponding to the random forest algorithm containing T decision trees.

[0101] When each decision tree splits at each node, randomly select k candidate features (k < K) from the K features. Based on the selected k features, calculate all possible split point metrics using binary numbers, and select the optimal split point according to the principle of minimizing error.

[0102] In the random forest, the above steps are independently performed for all T decision trees. Each decision tree outputs a fatigue status label for the sample, and the final result is determined by the majority vote of all decision trees, which is the output of the final random forest algorithm.

[0103] More specifically, in this embodiment, according to the joint angle offset, such as the knee joint angle attenuation ≥ 15%; and the force-position feature, the jump height decline rate ≥ 20%; the fatigue level is divided, and the fatigue levels include: no fatigue, mild fatigue, and severe fatigue. More specifically, it is changed that the joint angle change includes: taking the joint angle in the non-fatigued state as the baseline, calculating the ratio of the joint angle in different fatigue states to the baseline respectively. If the change exceeds 5%, it is regarded as moderate fatigue, and if the change exceeds 10%, it is regarded as severe fatigue. The force-position feature is the same. If the change in the ground reaction force and the jump height exceeds 2%, it is regarded as moderate fatigue, and if the change exceeds 5%, it is regarded as severe fatigue.

[0104] Specifically, the dataset is divided into 5 subsets (referred to as "folds"), and 4 folds are used as the training set and 1 fold is used as the validation set in turn. Finally, the generalization ability of the model is evaluated by the average performance of 5 experiments.

[0105] Specifically, a confusion matrix is used to visualize the recognition performance of the model. Each column of the confusion matrix represents the predicted class, and the total number of each column represents the number of data predicted as this class. Each row represents the true belonging class of the data, and the total number of data in each row represents the number of data instances of this class; the value in each column represents the number of true data predicted as this class; the importance of multi-modal features is ranked by OOB importance. By permuting or adding noise to the feature values, the change in the model performance is observed to quantify the feature importance. Approximately 36.8% of the samples in the training set of each decision tree are not selected, and this part of the samples is used as the validation set to evaluate the feature importance. After randomly permuting each feature, the difference in the prediction error of the model on the out-of-bag samples before and after the permutation is calculated. The greater the difference, the more important the feature is to the model.

[0106] Specifically, the fatigue state is identified for the test set according to the trained model, and the model outputs fatigue labels corresponding to different fatigue states. Embodiment Six

[0107] Based on any one of Embodiments Two to Five, a lower limb muscle fatigue state recognition system based on multi-modal feature fusion includes: a fatigue state recognition model, and a multi-modal sensor that provides bioelectrical signals, muscle morphology information, and force-position information to the fatigue state recognition model.

[0108] Specifically, the multi-modal sensor includes a bioelectrical signal sensor, a muscle morphology collector, and a force-position information acquisition device. The bioelectrical signal sensor, the muscle morphology collector, and the force-position information acquisition device are respectively interconnected with the fatigue state recognition model through a wireless electromyogram acquisition module, an infrared optical motion capture system, and a force platform system; it is used to implement a method for recognizing the fatigue state of lower limb muscles based on multi-modal feature fusion. The original electromyogram signal of the target muscle is obtained in real time through the wireless electromyogram acquisition module. The three-dimensional coordinates of the Mark points are captured by the infrared optical motion capture system, and the muscle cross-sectional circle radius, surface stretch length, and the change of the Mark point triangle area are dynamically calculated. The force platform data and the motion capture system achieve time synchronization through hardware triggering.

[0109] Working principle:

[0110] The present invention provides a method and system for recognizing the fatigue state of lower limb muscles based on multi-modal feature fusion, which have the characteristics of high detection accuracy, fast dynamic response, and strong anti-interference ability.

[0111] When the method and system for recognizing the fatigue state of lower limb muscles based on multi-modal feature fusion of the present invention are applied to the lower limbs:

[0112] According to the positions of the main fatigued muscles in the lower limbs, electromyogram electrodes are pasted on the surfaces of the bilateral rectus femoris muscles; according to the requirements of muscle cross-section and muscle surface strain, motion capture marker points are pasted on the surface of the rectus femoris muscle at equal intervals horizontally; according to the requirements of countermovement jumps, marker points are pasted on various parts of the lower limbs.

[0113] The electromyogram signals of the target muscles are collected according to the electromyogram sensors; the three-dimensional coordinates of the marker points pasted on the target muscles are collected in real time according to the motion capture system; the ground reaction forces during countermovement jumps in different fatigue states are collected according to the force platform.

[0114] Based on the electromyography (EMG) signals, preprocess and extract EMG features including root mean square, median frequency, and sample entropy; based on the three-dimensional coordinates and the three-dimensional force platform, extract muscle morphological features including cross-sectional circle radius and muscle surface stretching length, geometric features including the triangle area of marker points and joint angles, and force-position information including the ground reaction force of countermovement jump and the maximum height of countermovement jump, to form a multi-modal feature vector.

[0115] Based on the above multi-modal feature vector, design a muscle fatigue recognition model based on random forest, where the EMG features, muscle morphological features, and the triangle area of marker points are used as the model inputs; the joint angles and force-position information are used as indicators of fatigue states, and are verified and labeled for different fatigue states.

[0116] The above specific implementation manners are specific supports for the proposed solution idea of the present invention, and the protection scope of the present invention cannot be limited thereby. Any equivalent change or equivalent modification made on the basis of this technical solution according to the technical idea proposed by the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. A method for identifying the fatigue state of lower limb muscles based on multi-modal feature fusion, characterized in that , including the following steps: Step S1, arranging multimodal sensors on the object to be measured, and collecting signals through the multimodal sensors; The signal collection includes: obtaining bioelectrical signals, muscle morphology information, and force-position information through the multimodal sensors; Step S2, obtaining the fatigue state through the fatigue state recognition model according to the bioelectrical signals, muscle morphology information, and force-position information; specifically including: extracting myoelectric features from the bioelectrical signals, where the myoelectric features and muscle morphology information are used as model inputs, and the force-position information is used as an index of the fatigue state, verifying and marking different fatigue states; The muscle morphology feature extraction steps include: According to the bioelectrical signal sensor in the multimodal sensor, using an infrared optical motion capture system to capture the three-dimensional coordinates of the marked points set on the object to be measured, and dynamically calculating the muscle cross-sectional circle radius, surface stretching length, and change in the triangular area of the marked points according to the three-dimensional coordinates of the marked points; The calculation steps of the change in the triangular area of the marked points include: based on Heron's formula, calculating the area A of the triangle through the side lengths of the triangle; ; , where a, b, and c are the three side lengths of a triangle; The calculation steps for the radius of the muscle cross-sectional circle include: According to the calculation formula for the circumradius R calculate the circumradius, which is the radius of the muscle cross-sectional circle; And / or, the calculation steps of the surface stretching length include: based on the radius of the muscle cross-sectional circle, calculating the central angle θ through the cosine theorem, and calculating the arc length according to the radius and the central angle, where the radius of the muscle cross-sectional circle: ; Central angle: , where R is the radius of the muscle cross-sectional circle and C is the chord length; Arc length: , where arc is the arc length, the arc length is the surface stretching length, that is, the muscle surface strain; the arc length is obtained by multiplying θ by R; The force-position joint feature extraction steps include: The peak ground reaction force Fz-max is obtained by finding the maximum ground reaction force during the reverse jump, including the following steps: By traversing the maximum ground reaction force during the reverse jump, the maximum value is regarded as the peak ground reaction force Fz-max; , where is the function of the ground reaction force changing with time in the vertical direction Z, that is, the vertical component value of the ground reaction force at time t, is directly read by the force-position information acquisition device of the multi-modal sensor; Calculating the countermovement jump height through the vertical displacement of the hip joint marked points; including the following steps: By traversing the highest point of the z-axis coordinate of the marked points during the reverse jump process, subtracting the z-axis coordinate at the resting state from the highest point value to obtain the reverse jump height , , where is the highest point of the z-axis coordinate of the marked points, is the z-axis coordinate at the resting state; The calculation method of the joint angle includes the following steps: Establishing an equivalent model of the object to be measured through three marked points arranged on the object to be measured, and calculating the angles on the x, y, and z planes between the relatively active parts according to the equivalent model on the object to be measured to obtain the joint angles of the object to be measured.

2. The method for identifying the lower limb muscle fatigue state based on multi-modal feature fusion according to claim 1, characterized in that: The multimodal sensors include bioelectrical signal sensors, muscle morphology collectors, and force-position information collection devices; Collecting bioelectrical signals on the object to be measured through the bioelectrical signal sensors, collecting muscle morphology information on the object to be measured through the muscle morphology collectors, and collecting force-position information on the object to be measured through the force-position information collection devices.

3. The lower limb muscle fatigue state recognition method based on multi-modal feature fusion according to claim 2, characterized in that: Extracting myoelectric features from the bioelectrical signals through the fatigue state recognition model, and the myoelectric feature extraction steps include: Preprocessing the bioelectrical signals collected by the multimodal sensors; Then obtaining time-domain features, frequency-domain features, and non-linear features according to the preprocessed bioelectrical signals.

4. The method for recognizing the lower limb muscle fatigue state based on multi-modal feature fusion according to claim 3, wherein: The time-domain features include the root mean square RMS, and the root mean square RMS includes: , where X i is the voltage value of the i-th discrete sampling point in the EMG signal sequence; N is the total number of discrete data points within the EMG signal time window, and is calculated for the surface EMG signal sEMG signal with a length of N. And / or, the frequency domain feature includes a median frequency MF, and the median frequency MF includes: performing a Fourier transform on the preprocessed signal and calculating the power spectral density of the signal ; , where ω is the signal frequency.

5. The method for identifying the lower limb muscle fatigue state based on multi-modal feature fusion according to claim 4, wherein: The non-linear features include sample entropy SampEn, and the sample entropy SampEn includes: For a time series \(x(n)=\{x(1), x(2),\cdots, x(N)\}\) of length \(N\), given the embedding dimension \(m\), the original time series forms a vector group \(X\) of length \(N - m+1\) m (i) X m (i) = {x(i), x(i + 1),..., x(i + m - 1)}, where 1 ≤ i ≤ N - m + 1; Define d[X m (i), X m (j)] as the maximum distance between the corresponding vectors in two different vector groups; n is the position index of the data points in the time series, representing the nth time point; d[X m (i),X m (j)] = max(|x(i + k)-x(j + k)|), where 0 ≤ k ≤ m - 1, 1 ≤ i, j ≤ N - m + 1, and i ≠ j; Given r, r is the similarity tolerance; Defined as the ratio of the number of d[X m (i),X m (j)] ≤ r to the total number of vectors N - m + 1 is: ; where B i (r) is the number of d[X m (i),X m (j)] ≤ r; B i (r)=num{d[X m (i),X m (j)] ≤ r}, The average value of is expressed as , applicable to all i, ; When the embedding dimension is m + 1, the number of d[X m+1 (i),X m+1 (j)] ≤ r divided by the total number of vectors N - m is denoted as , , where A i (r) is the number of d[X m+1 (i),X m+1 (j)] ≤ r; A i (r) = num{d[X m+1 (i),X m+1 (j)] ≤ r}, and the average value of A i m (r) is expressed as D m+1 (r), applicable to all i, ; the sample entropy of the time series is: .

6. The method for identifying the lower limb muscle fatigue state based on multi-modal feature fusion according to claim 5, wherein: The fatigue state recognition model includes a random forest algorithm, and the random forest algorithm includes: the basic unit uses a decision tree, and the Gini coefficient is used to select the optimal splitting feature and threshold; The fatigue state recognition method of the fatigue state recognition model includes: dividing the fatigue level according to the joint angle offset and force-position features, and generating labels; Evaluating the model performance through cross-validation and outputting the fatigue state.

7. A lower limb muscle fatigue state recognition system based on multi-modal feature fusion, characterized in that: Including: A fatigue state recognition model, and a multimodal sensor that provides bioelectrical signals, muscle morphology information, and force-position information to the fatigue state recognition model; The multimodal sensors include bioelectrical signal sensors, muscle morphology collectors, and force-position information collection devices, and the bioelectrical signal sensors, muscle morphology collectors, and force-position information collection devices are respectively interconnected with the fatigue state recognition model through a wireless myoelectric acquisition module, an infrared optical motion capture system, and a force platform system; A method for recognizing the lower limb muscle fatigue state based on multi-modal feature fusion according to any one of claims 1-6.

Images