Lower limb muscle fatigue state identification method and system based on multi-modal feature fusion

Abstract

The invention discloses a lower limb muscle fatigue state recognition method and system based on multi-modal feature fusion, and the method comprises the following steps: S1, arranging a multi-modal sensor on a to-be-detected object, and carrying out the signal collection through the multi-modal sensor; bioelectricity signals, muscle form information and force position information are obtained through a multi-mode sensor; and S2, obtaining a fatigue state through a fatigue state recognition model according to the bio-electricity signal, the muscle form information and the force position information. The invention discloses a lower limb muscle fatigue state recognition method and system based on multi-modal feature fusion. The method and system have the advantages of being high in detection precision, fast in dynamic response and high in anti-interference performance.

Description

Technical Field

[0001] The present invention relates to the technical field of multimodal feature recognition, and particularly relates to a method and system for recognizing the fatigue state of lower limb muscles based on multimodal feature fusion. Background Art

[0002] Detecting muscle fatigue has far-reaching significance in the fields of sports science, rehabilitation medicine, occupational health, etc. Its real-time performance and accuracy are crucial for individual health management and optimization of sports performance. Existing muscle fatigue detection methods have the following defects: 1. Limitation of single data source: Existing technologies mostly rely on a single biological signal (such as surface electromyogram signal sEMG, heart rate, etc.), which is vulnerable to noise interference. High-frequency noise may cause feature loss or a single feature of joint angle, and cannot reflect the coupling relationship between the dynamic changes of muscle morphology and mechanical state; 2. Insufficient analysis of muscle characteristics: Existing technologies lack the ability to capture the fatigue assessment of continuous repetitive movements in real time and dynamically, and do not consider the influence of muscle cross-section deformation and muscle deformation on fatigue. Bioelectric signal sensors are vulnerable to motion artifacts and environmental interference; 3. Poor model generalization: The single feature dimension leads to insufficient classification accuracy of machine learning models for complex fatigue states. Summary of the Invention

[0003] The present invention overcomes the deficiencies of the prior art and provides a method and system for recognizing the fatigue state of lower limb muscles based on multimodal feature fusion, which has the characteristics of high detection accuracy, fast dynamic response, and strong anti-interference ability.

[0004] To achieve the above object, the technical solution adopted by the present invention is: A method for recognizing the fatigue state of lower limb muscles based on multimodal feature fusion, comprising the following steps: Step S1, arranging multimodal sensors on the object to be measured, and collecting signals through the multimodal sensors; the signal collection includes: obtaining bioelectric signals, muscle morphology information, and force-position information through the multimodal sensors; Step S2, obtaining the fatigue state through a fatigue state recognition model according to the bioelectric signals, muscle morphology information, and force-position information.

[0005] In a preferred embodiment of the present invention, the multimodal sensors include bioelectric signal sensors, muscle morphology collectors, and force-position information collection devices; Collecting bioelectric signals on the object to be measured through bioelectric signal sensors, collecting muscle morphology information on the object to be measured through muscle morphology collectors, and collecting force-position information on the object to be measured through force-position information collection devices.

[0006] In a preferred embodiment of the present invention, myoelectric feature extraction is performed on the bioelectrical signal through a fatigue state recognition model, and the myoelectric feature extraction steps include: Preprocess the bioelectrical signal collected by the multimodal sensor; Then obtain time-domain features, frequency-domain features, and non-linear features according to the preprocessed bioelectrical signal.

[0007] In a preferred embodiment of the present invention, the time-domain features include the root mean square (RMS), and the root mean square (RMS) includes: , where X i is the voltage value of the i-th discrete sampling point in the myoelectric signal sequence; N is the total number of discrete data points within the time window of the myoelectric signal, and it is calculated for the surface electromyogram (sEMG) signal with a length of N; And / or, the frequency-domain features include the median frequency (MF), and the median frequency (MF) includes: performing a Fourier transform on the preprocessed signal and calculating the power spectral density of the signal ; , where ω is the signal frequency.

[0008] In a preferred embodiment of the present invention, the non-linear features include sample entropy (SampEn), and the sample entropy (SampEn) includes: For a time series x(n) = {x(1), x(2),..., x(N)} with a length of N, given the embedding dimension m, the original time series forms a vector group X m (i), X m (i) = {x(i), x(i + 1),..., x(i + m - 1)}, where 1 ≤ i ≤ N - m + 1; define d[X m (i), X m (j)] as the maximum distance between the corresponding vectors in two different vector groups; n is the position index of the data points in the time series, representing the n-th time point; d[X m (i), X m (j)] = max(|x(i + k) - x(j + k)|), where 0 ≤ k ≤ m - 1, 1 ≤ i, j ≤ N - m + 1, and i is not equal to j; given r, r is the similarity tolerance; Define as the ratio of the number of d[X m (i), X m (j)] ≤ r to the total number of vectors N - m + 1: ; where B i (r) is d[X m (i), X m(j)]; the number of B ≤ r i (r) = num{d[X m (i), X m (j)] ≤ r}, The average value is denoted as , applicable to all i, ; When the embedding dimension is m + 1, the ratio of the number of d[X m+1 (i), X m+1 (j)] ≤ r to the total number of vectors N - m is denoted as , , where A i (r) is the number of d[X m+1 (i), X m+1 (j)] ≤ r; A i (r) = num{d[X m+1 (i), X m+1 (j)] ≤ r}, A i m (r) The average value is denoted as D m+1 (r), applicable to all i, ; The sample entropy of the time series is: .

[0009] In a preferred embodiment of the present invention, muscle morphology feature extraction is performed on muscle morphology information through a fatigue state recognition model. The muscle morphology feature extraction steps include: According to the bioelectric signal sensor in the multimodal sensor, the infrared optical motion capture system captures the three-dimensional coordinates of the marked points set on the object to be measured, and dynamically calculates the muscle cross-sectional circle radius, surface stretching length, and the change in the triangular area of the marked points according to the three-dimensional coordinates of the marked points.

[0010] In a preferred embodiment of the present invention, the calculation steps of the change in the triangular area of the marked points include: based on Heron's formula, calculating the area A of the triangle through the side lengths of the triangle; ; , where a, b, and c are the three side lengths of the triangle; The calculation steps of the muscle cross-sectional circle radius include: calculating the circumradius according to the formula for the circumradius R The circumradius is the muscle cross-sectional circle radius; And / or, the calculation steps of the surface stretching length include: based on the muscle cross-sectional circle radius, calculating the central angle θ through the cosine theorem, and calculating the arc length according to the radius and the central angle, where the muscle cross-sectional circle radius: ; The central angle: , where R is the radius of the muscle cross-section circle, C is the chord length, and the arc length: , where arc is the arc length, and the surface stretching length is obtained by multiplying the radian value of the central angle by the radius of the muscle surface strain.

[0011] In a preferred embodiment of the present invention, force-position joint features are extracted from the force-position information through a fatigue state recognition model. The steps for extracting the force-position joint features include: The peak ground reaction force Fz-max is obtained by finding the maximum ground reaction force during the reverse jump process, including the following steps: by traversing the maximum ground reaction force during the reverse jump process, the maximum value is regarded as the peak ground reaction force Fz-max; , where, is the function of the ground reaction force varying with time in the vertical direction Z, that is, the vertical component value of the ground reaction force at time t, is directly read by the force-position information acquisition device of the multi-modal sensor; The reverse jump height is calculated by the vertical displacement of the hip joint marker point; the steps include: By traversing the highest point of the marker point z-axis coordinate during the reverse jump process, subtracting the z-axis coordinate at the resting state from the highest point value to obtain the reverse jump height , , where, is the highest point of the marker point z-axis coordinate, is the z-axis coordinate at the resting state; The calculation method of the joint angle includes the following steps: An equivalent model of the object to be measured is established through three marker points arranged on the object to be measured, and the angles on the x, y, and z planes between the relatively active parts are calculated according to the equivalent model on the object to be measured to obtain the joint angle of the object to be measured.

[0012] In a preferred embodiment of the present invention, the fatigue state recognition model includes a random forest algorithm. The random forest algorithm includes: the basic unit uses a decision tree, and the Gini coefficient is used to select the optimal splitting feature and threshold; The fatigue state recognition method of the fatigue state recognition model includes: dividing the fatigue level according to the joint angle offset and the force-position characteristics, and generating labels; Evaluating the model performance through cross-validation and outputting the fatigue state.

[0013] In a preferred embodiment of the present invention, a lower limb muscle fatigue state recognition system based on multi-modal feature fusion includes: a fatigue state recognition model, and a multi-modal sensor that provides bioelectrical signals, muscle morphology information, and force-position information to the fatigue state recognition model; The multimodal sensor includes a bioelectrical signal sensor, a muscle morphology collector, and a force-position information acquisition device. The bioelectrical signal sensor, the muscle morphology collector, and the force-position information acquisition device are respectively interconnected with the fatigue state recognition model through a wireless electromyography acquisition module, an infrared optical motion capture system, and a force platform system; It is used to implement a method for recognizing the fatigue state of lower limb muscles based on multimodal feature fusion.

[0014] The present invention solves the defects existing in the technical background, and the beneficial technical effects of the present invention are: A method and system for recognizing the fatigue state of lower limb muscles based on multimodal feature fusion have the characteristics of high detection accuracy, fast dynamic response, and strong anti-interference ability.

[0015] Multimodal feature fusion further improves the fatigue recognition accuracy compared with the single electromyography method.

[0016] The real-time synchronous acquisition of muscle cross-sectional morphology and mechanical parameters realizes the rapid update of the fatigue state.

[0017] The introduction of multiple motion features reduces the interference of artifacts on the electromyography signal. Brief Description of the Drawings

[0018] The present invention will be further described below with reference to the drawings and embodiments.

[0019] Figure 1 It is a schematic diagram of the arrangement of multimodal sensors on the surface of the rectus femoris muscle in the preferred embodiment of the present invention; Figure 2 It is a schematic diagram of the arrangement of multimodal sensors at each joint during a countermovement jump in the preferred embodiment of the present invention Figure 1 (front); Figure 3 It is a schematic diagram of the arrangement of multimodal sensors at each joint during a countermovement jump in the preferred embodiment of the present invention Figure 2 (back); Figure 4 It is a schematic diagram of the working process of a method and system for recognizing the fatigue state of lower limb muscles based on multimodal feature fusion in the preferred embodiment of the present invention; Figure 5 It is a schematic diagram of the working process of using a random forest fatigue recognition model to recognize the fatigue state in a method and system for recognizing the fatigue state of lower limb muscles based on multimodal feature fusion in the preferred embodiment of the present invention; Figure 6 It is a curve graph of dynamic radius calculation in a method and system for recognizing the fatigue state of lower limb muscles based on multimodal feature fusion in the preferred embodiment of the present invention; Figure 7Schematic diagram of dynamic radius calculation in the lower limb muscle fatigue state recognition method and system based on multi-modal feature fusion according to the preferred embodiment of the present invention; Figure 8 Schematic diagram of electromyogram feature (taking root mean square as an example) calculation in the lower limb muscle fatigue state recognition method and system based on multi-modal feature fusion according to the preferred embodiment of the present invention; Figure 9 Schematic diagram of the calculation process of muscle cross-sectional circle radius, arc length and triangular area in the lower limb muscle fatigue state recognition method and system based on multi-modal feature fusion according to the preferred embodiment of the present invention; Figure 10 Schematic diagram of obtaining the maximum ground reaction force in the lower limb muscle fatigue state recognition method and system based on multi-modal feature fusion according to the preferred embodiment of the present invention; Figure 11 Schematic diagram of obtaining the maximum jump height in the lower limb muscle fatigue state recognition method and system based on multi-modal feature fusion according to the preferred embodiment of the present invention; Figure 12 Schematic diagram of obtaining the joint angle change amount in the lower limb muscle fatigue state recognition method and system based on multi-modal feature fusion according to the preferred embodiment of the present invention; Figure 13 Schematic diagram of the random forest algorithm process according to the preferred embodiment of the present invention. Detailed implementation manners

[0020] Now, the present invention will be further described in detail with reference to the accompanying drawings and embodiments. These drawings are all simplified schematic diagrams, only illustrating the basic structure of the present invention in a schematic manner, so they only show the components related to the present invention.

[0021] It should be noted that if there are directional indications (such as up, down, bottom, top, etc.) involved in the embodiments of the present invention, the directional indications are only used to explain the relative positional relationship and movement conditions between components in a specific posture. If the specific posture changes, the directional indications will also change accordingly. The terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. Unless otherwise clearly defined and limited, the terms "set", "connected" and "connected to" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be directly connected, or indirectly connected through an intermediate medium, and it can be the internal communication of two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations. Embodiment 1

[0022] As Figures 1 - 13 shown, a method for identifying the fatigue state of lower limb muscles based on multi-modal feature fusion includes the following steps: Step S1, arrange multi-modal sensors on the object to be measured, and collect signals through the multi-modal sensors; the signal collection includes: obtaining bioelectrical signals, muscle morphology information, and force-position information through the multi-modal sensors; Step S2, obtain the fatigue state through the fatigue state recognition model according to the bioelectrical signals, muscle morphology information, and force-position information.

[0023] Specifically, the multi-modal sensors include bioelectrical signal sensors, muscle morphology collectors, and force-position information collection devices; collect bioelectrical signals on the object to be measured through the bioelectrical signal sensors, collect muscle morphology information on the object to be measured through the muscle morphology collectors, and collect force-position information on the object to be measured through the force-position information collection devices.

[0024] Specifically, the myoelectric feature extraction is performed on the bioelectrical signals through the fatigue state recognition model, and the myoelectric feature extraction steps include: preprocessing the bioelectrical signals collected by the multi-modal sensors, and the preprocessing includes band-pass filtering, denoising, and segmentation; then obtain time-domain features, frequency-domain features, and non-linear features according to the preprocessed bioelectrical signals.

[0025] Specifically, the muscle morphology feature extraction is performed on the muscle morphology information through the fatigue state recognition model, and the muscle morphology feature extraction steps include: capturing the three-dimensional coordinates of the marked points set on the object to be measured by using an infrared optical motion capture system in the bioelectrical signal sensors in the multi-modal sensors, and dynamically calculating the muscle cross-sectional circle radius, surface stretching length, and change in the triangular area of the marked points according to the three-dimensional coordinates of the marked points. Embodiment 2

[0026] As Figures 1 - 13 shown, a method for identifying the fatigue state of lower limb muscles based on multi-modal feature fusion includes the following steps: Step S1, arrange multi-modal sensors on the object to be measured, and collect signals through the multi-modal sensors; the signal collection includes: obtaining bioelectrical signals, muscle morphology information, and force-position information through the multi-modal sensors; Step S2, obtain the fatigue state through the fatigue state recognition model according to the bioelectrical signals, muscle morphology information, and force-position information.

[0027] Specifically, the multi-modal sensors include bioelectrical signal sensors, muscle morphology collectors, and force-position information collection devices; collect bioelectrical signals on the object to be measured through the bioelectrical signal sensors, collect muscle morphology information on the object to be measured through the muscle morphology collectors, and collect force-position information on the object to be measured through the force-position information collection devices.

[0028] Specifically, the myoelectric feature extraction of bioelectrical signals is performed by a fatigue state recognition model. The myoelectric feature extraction steps include: preprocessing the bioelectrical signals collected by a multi-modal sensor, where the preprocessing includes band-pass filtering, denoising, and segmentation; then obtaining time-domain features, frequency-domain features, and non-linear features based on the preprocessed bioelectrical signals. More specifically, the preprocessing includes band-pass filtering, denoising, and segmentation processing. The preprocessing of the data is for the three-modal data of bioelectrical signals, muscle morphology information, and force-position information collected by a bioelectrical signal sensor (i.e., a wireless myoelectric acquisition module), a muscle morphology collector (i.e., an infrared optical motion capture system), and a force-position information acquisition device (i.e., a force platform system). Among them, the preprocessing includes band-pass filtering, denoising, and segmentation processing for the bioelectrical signals; and segmentation processing for the muscle morphology information and the force-position information. Among them, the band-pass filtering uses a Chebyshev type I filter (20 - 450 Hz); the denoising uses wavelet decomposition denoising, and the wavelet basis function is selected as Daubechies4 (db4), and the wavelet packet decomposition layer is 4.

[0029] Furthermore, the time-domain features include the root mean square RMS, and the root mean square RMS includes: , where X i is the voltage value of the i-th discrete sampling point in the myoelectric signal sequence.

[0030] Furthermore, the frequency-domain features include the median frequency MF, and the median frequency MF includes: performing a Fourier transform on the preprocessed signal to calculate the power spectral density P(ω) of the signal, , where ω is the signal frequency.

[0031] Furthermore, the non-linear features include the sample entropy SampEn, and the sample entropy SampEn includes: For a time series x(n) = {x(1), x(2),..., x(N)} of length N, given an embedding dimension m, the original time series forms a vector group X of length N - m + 1 m (i), X m (i) = {x(i), x(i + 1),..., x(i + m - 1)}, where 1 ≤ i ≤ N - m + 1; define d[X m (i), X m (j)] as the maximum distance between the corresponding vectors in two different vector groups; n is the position index of the data points in the time series, representing the n-th time point; d[X m (i), X m(j)] = max(|x(i + k) - x(j + k)|), where 0 ≤ k ≤ m - 1, 1 ≤ i, j ≤ N - m + 1, and i is not equal to j; given r, r is the similarity tolerance; is defined as the ratio of the number of d[X m (i), X m (j)] ≤ r to the total number of vectors N - m + 1: ; where B i (r) is the number of d[X m (i), X m (j)] ≤ r; B i (r) = num{d[X m (i), X m (j)] ≤ r}, The average value of is expressed as , applicable to all i, ; When the embedding dimension is m + 1, the ratio of the number of d[X m+1 (i), X m+1 (j)] ≤ r to the total number of vectors N - m is denoted as , , where A i (r) is the number of d[X m+1 (i), X m+1 (j)] ≤ r; A i (r) = num{d[X m+1 (i), X m+1 (j)] ≤ r}, A i m (r) The average value of is expressed as D m+1 (r), applicable to all i, ; The sample entropy of the time series is: .

[0032] Specifically, the muscle morphology feature extraction is performed on the muscle morphology information through the fatigue state recognition model. The muscle morphology feature extraction steps include: capturing the three-dimensional coordinates of the marked points set on the object to be measured by the infrared optical motion capture system according to the bioelectric signal sensor in the multi-modal sensor, and dynamically calculating the muscle cross-sectional circle radius, surface stretching length, and the change in the triangular area of the marked points based on the three-dimensional coordinates of the marked points.

[0033] Further, muscle morphological features are extracted from the muscle morphological information through a fatigue state recognition model. The muscle morphological feature extraction steps include: capturing the three-dimensional coordinates of the marked points set on the object to be measured by using an infrared optical motion capture system according to the bioelectric signal sensor in the multimodal sensor, and dynamically calculating the muscle cross-sectional circle radius, surface stretching length, and change in the triangular area of the marked points based on the three-dimensional coordinates of the marked points.

[0034] Further, the calculation steps for the change in the triangular area of the marked points include: based on Heron's formula, calculating the area A of the triangle through the lengths of the sides of the triangle, ; , where a, b, and c are the three sides of the triangle.

[0035] Further, the calculation steps for the muscle cross-sectional circle radius include: calculating the radius of the circumscribed circle according to the calculation formula of the circumscribed circle radius R The circumscribed circle radius calculated is the muscle cross-sectional circle radius.

[0036] Further, the calculation steps for the surface stretching length include: based on the muscle cross-sectional circle radius, calculating the central angle θ through the cosine theorem, and calculating the arc length according to the radius and the central angle. Among them, the muscle cross-sectional circle radius: ; The central angle: , where R is the muscle cross-sectional circle radius, C is the chord length, and the arc length: , where arc is the arc length, and the surface stretching length is obtained by multiplying the radian value of the central angle by the radius, which is the muscle surface strain.

[0037] Specifically, force-position joint features are extracted from the force-position information through a fatigue state recognition model. The force-position joint feature extraction steps include: Further, obtaining the peak value of the ground reaction force Fz-max by finding the maximum ground reaction force during the reverse jump includes the following steps: by traversing the maximum value of the ground reaction force during the reverse jump, regarding the maximum value as the peak value of the ground reaction force Fz-max, , where, is the function of the ground reaction force in the vertical direction Z changing with time.

[0038] Further, calculating the reverse jump height through the vertical displacement of the hip joint marked point; includes the following steps: By traversing the highest point of the z-axis coordinate of the marked point during the reverse jump, subtracting the z-axis coordinate at the resting state from the highest point value to obtain the reverse jump height , , where, is the highest point of the z-axis coordinate of the marked point, is the z-axis coordinate at the resting state; Further, the method for calculating the joint angle includes the following steps: An equivalent model of the object to be measured is established through three marking points arranged on the object to be measured, and the angles between relatively movable parts on the x, y, and z planes are calculated according to the equivalent model on the object to be measured, so as to obtain the joint angle of the object to be measured. Embodiment III

[0039] Based on Embodiment II, the fatigue state recognition model includes a random forest algorithm, and the random forest algorithm includes: the basic unit uses a decision tree, and the Gini coefficient is used to select the optimal splitting feature and threshold. Specifically, the number of decision trees is 300; the minimum number of leaf nodes is 10, and the maximum depth is 20. Embodiment IV

[0040] Based on Embodiment 3 or Embodiment 2, in this embodiment, the bioelectric signal sensor (electromyogram signal acquisition device) used is the ErgoLAB surface electromyogram measurement system, with a sampling frequency of 1000 Hz, including 8-channel acquisition electrodes, and supporting Bluetooth 5.0 wireless signal transmission. The muscle morphology collector (motion capture system) used is the Vicon optical motion capture system, model Vicon Mx-GIGA NET, and its hardware includes 16 Vicon MX cameras, a PC host, a Vicon Datastation ADCPatchPanel information conversion box, and MX special connecting wires, etc., with a sampling frequency of 100 Hz. The force-position information acquisition device (force platform) used is a three-dimensional force platform in the prior art, with six high-precision mechanical sensors built-in, which can collect the mechanical signals acting on the platform in real time and directly obtain the following mechanical parameters: Fx, Fy, Fz, Mx, My, Mz, three-dimensional force vector, center of pressure (COP), with a sampling frequency of 1000 Hz. More specifically, in this embodiment, the object to be measured is the lower limb. Therefore, in this embodiment, the electromyogram electrodes are arranged according to the anatomical positions of the main fatigued muscles of the lower limb (such as the bilateral rectus femoris muscles), and the electromyogram electrodes are pasted on the surface of the muscle belly; the arrangement of the motion capture marker points (Mark points): Based on the muscle cross-sectional strain distribution law of the rectus femoris muscle, reflective marker points are arranged at equal intervals horizontally on the muscle surface (spacing 40 mm), and at the same time, Mark points are pasted at the ankle, knee, and hip joints to synchronously capture the kinematic data of the countermovement jump; mechanical data acquisition: A pressure-sensitive area is set on the surface of the force platform to synchronously collect the three-dimensional ground reaction force during the countermovement jump. Among them, Fx: the force component along the X-axis direction, defined as the horizontal lateral force of the sensor (such as the front-back direction); Fy: the force component along the Y-axis direction, defined as the horizontal longitudinal force of the sensor (such as the left-right direction); Fz: the force component along the Z-axis direction, defined as the vertical direction force of the sensor (such as the up-down direction); Mx: the moment component around the X-axis, indicating the rotational effect of the object around the lateral axis of the sensor (such as torsion); My: the moment component around the Y-axis, indicating the rotational effect of the object around the longitudinal axis (such as tilt); Mz: the moment component around the Z-axis, indicating the rotational effect of the object around the vertical axis (such as rotation). Embodiment 5

[0041] Based on Embodiment 2, Embodiment 3, or Embodiment 4, the fatigue state recognition method of the fatigue state recognition model includes: dividing the fatigue level according to the joint angle offset and force-position characteristics, and generating labels.

[0042] Evaluate the model performance through cross-validation and output the fatigue state.

[0043] Specifically, the fatigue state recognition model includes a feature extraction module and a random forest algorithm including cross-validation; among them, the feature extraction module includes: electromyogram feature extraction, muscle morphology feature extraction, and force-position information extraction.

[0044] Random forest is an ensemble learning method that performs classification tasks by integrating multiple decision trees. For a dataset containing K samples, assuming each sample contains S features (the features are the above-mentioned electromyography features and muscle morphology features). Randomly draw K samples (with repetition allowed) from the dataset to form a training subset for a single decision tree. Repeat the sampling T times to generate T independent data subsets, corresponding to T decision trees in the random forest algorithm.

[0045] When each decision tree splits at each node, randomly select k candidate features (k < K) from the K features. Based on the selected k features, calculate all possible split point metrics using binary numbers, and select the optimal split point according to the principle of minimizing error.

[0046] In the random forest, the above steps are independently performed for all T decision trees. Each decision tree outputs a fatigue status label for the sample, and the final result is determined by the majority vote of all decision trees, which is the output of the final random forest algorithm.

[0047] More specifically, in this embodiment, according to the joint angle offset, such as the knee joint angle decay ≥ 15%; and the force-position feature, the jump height decline rate ≥ 20%; the fatigue level is divided, and the fatigue levels include: no fatigue, mild fatigue, and severe fatigue. More specifically, it is changed that the joint angle change includes: taking the joint angle in the non-fatigued state as the baseline, calculating the ratio of the joint angle in different fatigue states to the baseline respectively. If the change exceeds 5%, it is regarded as moderate fatigue, and if the change exceeds 10%, it is regarded as severe fatigue. The force-position feature is the same. If the change in the ground reaction force and the jump height exceeds 2%, it is regarded as moderate fatigue, and if the change exceeds 5%, it is regarded as severe fatigue.

[0048] Specifically, the dataset is divided into 5 subsets (referred to as "folds"), and 4 folds are used as the training set and 1 fold is used as the validation set in turn. Finally, the generalization ability of the model is evaluated by the average performance of 5 experiments.

[0049] Specifically, a confusion matrix is used to visualize the recognition performance of the model. Each column of the confusion matrix represents the predicted class, and the total number of each column represents the number of data predicted as this class. Each row represents the true belonging class of the data, and the total number of data in each row represents the number of data instances of this class; the value in each column represents the number of true data predicted as this class; the importance of multi-modal features is arranged by OOB importance. By permuting or disturbing the feature values with noise, observe the change in the model performance to quantify the feature importance. Approximately 36.8% of the samples in the training set of each decision tree are not selected, and this part of the samples is used as the validation set to evaluate the feature importance. After randomly permuting each feature, calculate the difference in the prediction error of the model on the out-of-bag samples before and after the permutation. The greater the difference, the more important the feature is to the model.

[0050] Specifically, the fatigue state is identified for the test set according to the trained model, and the model outputs fatigue labels corresponding to different fatigue states. Embodiment Six

[0051] Based on any one of Embodiments Two to Five, a lower limb muscle fatigue state recognition system based on multi-modal feature fusion includes: a fatigue state recognition model, and a multi-modal sensor that provides bioelectrical signals, muscle morphology information, and force-position information to the fatigue state recognition model.

[0052] Specifically, the multi-modal sensor includes a bioelectrical signal sensor, a muscle morphology collector, and a force-position information acquisition device. The bioelectrical signal sensor, the muscle morphology collector, and the force-position information acquisition device are respectively interconnected with the fatigue state recognition model through a wireless electromyogram acquisition module, an infrared optical motion capture system, and a force platform system; to implement a method for recognizing the fatigue state of lower limb muscles based on multi-modal feature fusion. The original electromyogram signals of the target muscle are obtained in real time through the wireless electromyogram acquisition module. The infrared optical motion capture system is used to capture the three-dimensional coordinates of the Mark points, and the muscle cross-sectional circle radius, surface stretch length, and Mark point triangle area change are dynamically calculated. The force platform data and the motion capture system are time-synchronized through hardware triggering.

[0053] Working principle: The present invention provides a method and system for recognizing the fatigue state of lower limb muscles based on multi-modal feature fusion, which has the characteristics of high detection accuracy, fast dynamic response, and strong anti-interference ability.

[0054] When the method and system for recognizing the fatigue state of lower limb muscles based on multi-modal feature fusion of the present invention are applied to the lower limbs: According to the positions of the main fatigued muscles in the lower limbs, electromyogram electrodes are pasted on the surfaces of the bilateral rectus femoris muscles; according to the requirements of muscle cross-section and muscle surface strain, motion capture marker points are pasted on the surface of the rectus femoris muscle at equal intervals horizontally; according to the requirements of a countermovement jump, marker points are pasted on various parts of the lower limbs.

[0055] The electromyogram signals of the target muscle are collected according to the electromyogram sensor; the three-dimensional coordinates of the marker points pasted on the target muscle are collected in real time according to the motion capture system; the ground reaction force of the countermovement jump under different fatigue states is collected according to the force platform.

[0056] According to the electromyogram signals, electromyogram features including root mean square, median frequency, and sample entropy are preprocessed and extracted; according to the three-dimensional coordinates and the three-dimensional force platform, muscle morphology features including cross-sectional circle radius and muscle surface stretch length, geometric features including marker point triangle area and joint angle, and force-position information including ground reaction force of the countermovement jump and maximum height of the countermovement jump are extracted to form a multi-modal feature vector.

[0057] Based on the above-mentioned multi-modal feature vectors, a muscle fatigue recognition model is designed based on random forest, where the electromyogram features, muscle morphology features, and the triangular area of the marker points are used as the model inputs; the joint angles and force-position information are used as indicators of the fatigue state, and the fatigue states are verified and labeled.

[0058] The above specific implementation manners are specific supports for the proposed solution idea of the present invention, and the protection scope of the present invention cannot be limited thereby. Any equivalent change or equivalent modification made on the basis of this technical solution according to the technical idea proposed by the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. A method for identifying lower limb muscle fatigue state based on multimodal feature fusion, characterized in that , including the following steps: Step S1, arranging a multimodal sensor on the object to be measured, and collecting signals through the multimodal sensor; Signal acquisition includes: obtaining bioelectric signals, muscle morphology information, and force position information through multimodal sensors; Step S2, obtaining the fatigue state through a fatigue state recognition model according to the bioelectric signal, muscle morphology information, and force position information.

2. The method for identifying lower limb muscle fatigue state based on multimodal feature fusion according to claim 1, characterized in that: The multimodal sensor includes a bioelectric signal sensor, a muscle morphology collector, and a force position information collection device; The bioelectric signal on the object to be tested is collected by the bioelectric signal sensor, the muscle morphology information on the object to be tested is collected by the muscle morphology collector, and the force position information on the object to be tested is collected by the force position information collection device.

3. The method for identifying lower limb muscle fatigue state based on multimodal feature fusion according to claim 2, characterized in that: The myoelectric feature extraction of the bioelectric signal is performed using a fatigue state recognition model, and the myoelectric feature extraction step includes: Preprocessing the bioelectric signals collected by multimodal sensors; Then, the time domain features, frequency domain features and nonlinear features are obtained based on the preprocessed bioelectric signals.

4. The method for identifying lower limb muscle fatigue state based on multimodal feature fusion according to claim 3 is characterized in that: The time domain characteristics include root mean square RMS, and the root mean square RMS includes: , where X i is the voltage value of the ith discrete sampling point in the EMG signal sequence; N is the total number of discrete data points in the EMG signal time window, which is calculated for the surface EMG signal (sEMG) with a length of N; And / or, the frequency domain feature includes a median frequency MF, and the median frequency MF includes: performing Fourier transform on the preprocessed signal and calculating the power spectrum density of the signal ; , where ω is the signal frequency.

5. The method for identifying lower limb muscle fatigue state based on multimodal feature fusion according to claim 4, characterized in that: The nonlinear feature includes sample entropy SampEn, and the sample entropy SampEn includes: For a time series x(n)={x(1),x(2),...,x(N)} of length N, given an embedding dimension m, the original time series forms a vector group X of length N-m+1 m (i) X m (i) = {x(i), x(i+1), ..., x(i+m-1)}, where 1≤i≤N-m+1; define d[X m (i),X m (j)] is the maximum distance between corresponding vectors in two different vector groups; n is the position index of the data point in the time series, indicating the nth time point; d[X m (i),X m (j)] = max(|x(i+k)-x(j+k)|), where 0≤k≤m-1, 1≤i, j≤N-m+1, i is not equal to j; given r, r is the similarity tolerance; Defined as d[X m (i),X m The ratio of the number of vectors with [(j)]≤r to the total number of vectors N-m+1 is: Among them, B i (r) is d[X m (i),X m (j)]≤r; B i (r)=num{d[X m (i),X m (j)]≤r}, The average value is expressed as , applicable to all i, ; When the embedding dimension is m+1, d[X m+1 (i),X m+1 The ratio of the number of vectors Nm to the total number of vectors Nm is denoted by , , where A i (r) is d[X m+1 (i),X m+1 (j)]≤r; A i (r)=num{d[X m+1 (i),X m+1 (j)]≤r},A i m The average value of (r) is expressed as D m+1 (r), applies to all i, ; The sample entropy of the time series is: .

6. The method for identifying lower limb muscle fatigue state based on multimodal feature fusion according to claim 5, characterized in that: The muscle morphology information is subjected to muscle morphology feature extraction through the fatigue state recognition model. The muscle morphology feature extraction steps include: The bioelectric signal sensor in the multimodal sensor uses an infrared optical motion capture system to capture the three-dimensional coordinates of the set marking points on the object to be tested, and the muscle cross-sectional circle radius, surface stretching length, and triangular area change of the marking points are dynamically calculated based on the three-dimensional coordinates of the marking points.

7. The method for identifying lower limb muscle fatigue state based on multimodal feature fusion according to claim 6, characterized in that: The step of calculating the change in the area of ​​the triangle of the marking point comprises: calculating the area A of the triangle by the length of each side of the triangle based on Heron's formula; ; , where a, b, c are the lengths of the three sides of the triangle; The step of calculating the radius of the muscle cross-section circle includes: according to the calculation formula of the circumscribed circle radius R Calculate the radius of the circumscribed circle, where the radius of the circumscribed circle is the radius of the muscle cross-section circle; And / or, the step of calculating the surface stretching length includes: calculating the center angle θ based on the radius of the muscle cross-section circle by the cosine theorem, and calculating the arc length according to the radius and the center angle, wherein the radius of the muscle cross-section circle is: ; Center angle: , where R is the radius of the muscle cross section, C is the chord length, and arc length: , where arc is the arc length, the surface stretch length is the muscle surface strain obtained by multiplying the arc value of the central angle by the radius.

8. The method for identifying lower limb muscle fatigue state based on multimodal feature fusion according to claim 7, characterized in that: The force-position joint feature is extracted from the force-position information by using the fatigue state recognition model, and the force-position joint feature extraction step includes: The peak value Fz-max of the ground reaction force is obtained by finding the maximum ground reaction force during the counter-jump, comprising the following steps: traversing the maximum value of the ground reaction force during the counter-jump, and taking the maximum value as the peak value Fz-max of the ground reaction force; ,in, is the function of the ground reaction force in the vertical direction Z changing with time, that is, the vertical component value of the ground reaction force at time t, Directly read by the force and position information acquisition device of the multi-modal sensor; Countermovement jump height is calculated by vertical displacement of hip joint markers; the following steps are involved: By traversing the highest point of the z-axis coordinate of the marked point during the reverse jump, the z-axis coordinate of the highest point is subtracted from the resting state to obtain the reverse jump height. , ,in, is the highest point of the z-axis coordinate of the marking point, is the z-axis coordinate in the resting state; The method for calculating the joint angle includes the following steps: An equivalent model of the object to be tested is established by arranging three marking points on the object to be tested, and the angles between relatively active parts on the x, y, and z planes are calculated according to the equivalent model on the object to be tested to obtain the joint angles of the object to be tested.

9. The method for identifying lower limb muscle fatigue state based on multimodal feature fusion according to claim 8, characterized in that: The fatigue state recognition model includes a random forest algorithm, which includes: a basic unit adopts a decision tree and uses the Gini coefficient to select the optimal splitting feature and threshold; The fatigue state recognition method of the fatigue state recognition model includes: dividing the fatigue level according to the joint angle offset and the force position characteristics, and generating a label; The model performance is evaluated through cross-validation and the fatigue status is output.

10. A lower limb muscle fatigue state recognition system based on multimodal feature fusion, characterized in that: include: A fatigue state recognition model, and a multimodal sensor that provides bioelectric signals, muscle morphology information, and force position information to the fatigue state recognition model; The multimodal sensor includes a bioelectric signal sensor, a muscle morphology collector, and a force position information collection device, and the bioelectric signal sensor, muscle morphology collector, and force position information collection device are interconnected with the fatigue state recognition model through a wireless electromyography collection module, an infrared optical motion capture system, and a force platform system respectively; A method for identifying lower limb muscle fatigue status based on multimodal feature fusion, used to implement any one of claims 1-9.

Images