Muscle fatigue detection method based on multi-modal physiological information graph and related device

Abstract

The invention provides a muscle fatigue detection method and device based on a multi-modal physiological information graph, electronic equipment and a storage medium, and relates to the field of biological information. The method comprises the steps that space-time synchronization signals of a target muscle group are acquired, and the space-time synchronization signals comprise electromyographic signals and muscle oxygen signals; performing data preprocessing on the space-time synchronization signal to extract a target characteristic parameter reflecting a muscle fatigue state; performing visualization processing based on the target characteristic parameters to obtain a multi-modal physiological information graph; and inputting the multi-modal physiological information graph as input data into a target detection model to obtain a muscle fatigue classification result. Therefore, the fusion of the multi-modal physiological information graph is realized based on the acquired space-time synchronization signal and the target detection model, then the identification of muscle fatigue is completed, the objectivity and accuracy of waist and back muscle fatigue detection and classification are finally improved, and an objective and effective prevention means is provided for preventing chronic lumbago and acute muscle injury.

Description

Technical Field

[0001] This application relates to the field of bioinformatics, and more specifically, to a method, apparatus, electronic device, and storage medium for detecting muscle fatigue based on multimodal physiological information maps. Background Technology

[0002] Muscle fatigue is a common physiological phenomenon during exercise training and rehabilitation, defined as a temporary decrease in the ability of muscles to generate and maintain force during sustained or repetitive muscle activity. Lower back muscle fatigue is mainly manifested as decreased strength, reduced endurance, and impaired motor control in the muscles of the lower back, lumbosacral region, and sacroiliac region. Excessive fatigue of the lower back muscles is a significant factor leading to chronic lower back pain and muscle injury. Timely and accurate assessment of lower back muscle fatigue is crucial for preventing chronic lower back pain and optimizing work intensity.

[0003] Currently, methods for detecting lower back muscle fatigue mainly rely on single-modal recognition or manual extraction of multimodal physiological signal features. However, existing detection methods have the following problems: single-modal recognition lacks consideration of the correlation between modalities, and single data analysis can easily lead to low accuracy in muscle fatigue classification results.

[0004] As can be seen from the above, there is an urgent need for a muscle fatigue detection method based on multimodal physiological information maps to solve the above problems. Summary of the Invention

[0005] This application provides a method, device, electronic device, and storage medium for muscle fatigue detection based on multimodal physiological information maps. These solutions address the shortcomings of existing methods, such as the lack of analysis revealing spatial coordination and temporal-frequency complementarity between different muscle modalities, and the lack of accuracy in muscle fatigue detection due to the lack of fusion of multimodal physiological information maps. This application will use lumbar and back muscle fatigue detection as an example to describe the solution in detail. The technical solutions are as follows: According to one aspect of this application, a muscle fatigue detection method based on a multimodal physiological information map is proposed. The method includes: acquiring spatiotemporal synchronization signals of a target muscle group, wherein the spatiotemporal synchronization signals include electromyographic signals and muscle oxygenation signals; performing data preprocessing on the spatiotemporal synchronization signals to extract target feature parameters reflecting muscle fatigue state; performing visualization processing based on the target feature parameters to obtain a multimodal physiological information map, wherein the multimodal physiological information map is obtained by transforming the discretely arranged spatiotemporal synchronization signals into a visualized feature topographic map containing spatial distribution information; and inputting the multimodal physiological information map as input data into a target detection model to obtain a muscle fatigue classification result, wherein the target detection model is constructed based on a two-stream convolutional hybrid attention fusion network.

[0006] According to one aspect of this application, a muscle fatigue detection device based on a multimodal physiological information map is proposed. The device includes a signal acquisition module, a preprocessing module, a visualization processing module, and a result output module, wherein: the signal acquisition module is used to acquire spatiotemporal synchronization signals of a target muscle group, wherein the spatiotemporal synchronization signals include electromyographic signals and muscle oxygenation signals; the preprocessing module is used to perform data preprocessing on the spatiotemporal synchronization signals to extract target feature parameters reflecting muscle fatigue state; the visualization processing module is used to perform visualization processing based on the target feature parameters to obtain a multimodal physiological information map, wherein the multimodal physiological information map is obtained by transforming the discretely arranged spatiotemporal synchronization signals into a visualized feature topographic map containing spatial distribution information; the result output module is used to input the multimodal physiological information map as input data into a target detection model to obtain a muscle fatigue classification result, wherein the target detection model is constructed based on a two-stream convolutional hybrid attention fusion network.

[0007] According to one aspect of this application, an electronic device includes at least one processor and at least one memory, wherein program instructions or code are stored in the memory; the program instructions or code are loaded and executed by the processor, causing the electronic device to implement the muscle fatigue detection method based on multimodal physiological information maps as described above.

[0008] According to one aspect of this application, a storage medium stores program instructions or code thereon, which are loaded and executed by a processor to implement the muscle fatigue detection method based on multimodal physiological information maps as described above.

[0009] According to one aspect of this application, a computer program product includes program instructions or code stored in a storage medium. The processor of an electronic device reads the program instructions or code from the storage medium, loads and executes the program instructions or code, causing the electronic device to implement the muscle fatigue detection method based on multimodal physiological information maps as described above.

[0010] The beneficial effects of the technical solution provided in this application are: In the above technical solution, firstly, spatiotemporal synchronization signals of the target muscle group are acquired, including electromyographic (EMG) signals and muscle oxygenation signals. Then, the spatiotemporal synchronization signals are preprocessed to extract target feature parameters reflecting muscle fatigue. Further, visualization processing is performed based on the target feature parameters to obtain a multimodal physiological information map. Finally, the multimodal physiological information map is used as input data to a target detection model to obtain muscle fatigue classification results. Thus, by fusing the acquired spatiotemporal synchronization signals and the target detection model, the multimodal physiological information map is fused, thereby completing the identification of muscle fatigue. Ultimately, this improves the objectivity and accuracy of detecting and classifying lower back muscle fatigue, providing an objective and effective preventative measure for chronic low back pain and acute muscle injuries. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0012] Figure 1 This is a flowchart illustrating a muscle fatigue detection method based on multimodal physiological information maps according to an exemplary embodiment. Figure 2 This is an electromyographic feature topographic map generated according to an array design diagram and example shown in an exemplary embodiment; Figure 3 This is a schematic diagram illustrating the fatigue classification process of a target detection model according to an exemplary embodiment; Figure 4 This is a schematic diagram illustrating the accuracy of experimental results according to an exemplary embodiment; Figure 5 This is a schematic diagram of a confusion matrix for fatigue identification of subject 3, according to an exemplary embodiment. Figure 6 This is a structural block diagram of a muscle fatigue detection device based on a multimodal physiological information map, according to an exemplary embodiment. Figure 7 This is a structural block diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0013] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0014] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0015] The following is an introduction and explanation of the relevant academic background involved in this application: Muscle fatigue is a common physiological phenomenon during sports training and rehabilitation, defined as a temporary decrease in the ability of muscles to generate and maintain force during sustained or repetitive muscle activity. Lower back muscle fatigue is mainly manifested as decreased strength, reduced endurance, and impaired motor control in the muscles of the lower back, lumbosacral region, and sacroiliac region.

[0016] If back muscle fatigue is not relieved or intervened in time during exercise or labor, it can easily accumulate and induce or aggravate back pain and muscle injury. According to statistics, hundreds of thousands of people in my country are injured each year due to non-benign muscle fatigue caused by improper exercise methods or improper control of exercise volume.

[0017] Meanwhile, lower back pain and muscle injuries caused by excessive fatigue of the lower back muscles are more common in occupational groups that require prolonged forward bending, heavy lifting, or repetitive labor, such as those in manufacturing, logistics and transportation, and medical care. Persistent lower back muscle fatigue not only weakens an individual's work capacity and activity tolerance, but may also lead to sleep disorders, emotional problems, and a decline in quality of life, thus having a long-term impact on the patient, the family's care burden, and social medical and productivity.

[0018] The following is a comparative analysis of publicly available patents related to muscle fatigue detection in recent years. Please refer to Table 1 for details: Table 1 Submitting Unit / Individual Patent Name / Solution Core content Defects exist Guangdong Mousse Healthy Sleep Co., Ltd. A method, apparatus, equipment, and medium for detecting lumbar muscle fatigue (Application No.: 202411801963.1) Acquire electromyography, temperature data, and subjective fatigue levels; weight these data to determine the target fatigue level; and adjust the smart mattress mode accordingly. The weights are set arbitrarily without an adaptive mechanism, requiring subjective evaluation; there is no physiological information map; and there are no advanced fusion methods. Qilu University of Technology Method, device and system for detecting lumbar fatigue based on physiological parameters (Application No.: 202110333780.1) Surface electromyography (EMG) signals are collected, preprocessed, and then time-domain and frequency-domain features are extracted. These features are then combined for analysis to determine fatigue. It only uses unimodal electromyography, lacks multimodal fusion, and does not utilize deep learning to mine multidimensional features, resulting in poor robustness. Hefei Institutes of Physical Science, Chinese Academy of Sciences A method for detecting multiple types of muscle fatigue based on feature extraction and GRU deep learning model (Application No.: 202111371113.9) Acquire back electromyography (EMG) signals, clean the data, extract features using a sliding window method, and train the model using a GRU algorithm. Relying solely on electromyography and manual feature extraction, without physiological information maps or deep learning fusion networks. Hong Kong Polytechnic University Shenzhen Research Institute A method and apparatus for detecting muscle fatigue level (Application No.: 201811021478.7) Biometric signals were acquired, including the mean amplitude of hemoglobin wavelets and the median frequency of electromyography, and the fatigue index was obtained through linear weighting. Without a designated target muscle group, without multi-channel simultaneous acquisition, without physiological information maps and deep learning fusion, the accuracy is limited. Harbin Institute of Technology (Shenzhen) (Liu Honghai et al.) A method and system for monitoring muscle fatigue based on electromyographic signals and blood oxygen concentration (Application No.: 202411088742.4) Calculate electromyographic fatigue factor and blood oxygen fatigue factor, and output fatigue status. It relies on fixed handcrafted features and weighted fusion, lacks adaptive weights, and has limited robustness and generalization ability. As can be seen from the defect analysis in Table 1, current methods for detecting muscle fatigue, such as those for detecting lower back muscle fatigue, mainly rely on single-modal recognition or manual extraction of multimodal physiological signal features. These methods struggle to systematically characterize the spatiotemporal coupling relationship between different modalities and lack adaptive fusion strategies, thus limiting detection accuracy and generalization ability.

[0019] For example, currently, the most common method is to determine the fatigue of the lower back muscles by using electromyographic signal features. In multimodal joint detection, using only manual extraction of multimodal features or using weighted methods for modality fusion results in insufficient spatial resolution, and the fusion method is relatively crude, failing to fully utilize the complementary information between different modalities, thus resulting in insufficient accuracy.

[0020] To improve the objectivity and accuracy of lumbar and back muscle fatigue detection and classification, and to prevent chronic low back pain and lumbar and back muscle injuries, this application proposes a muscle fatigue detection method and related device based on multimodal physiological information maps. The core of this method is to achieve lumbar and back muscle fatigue detection based on multimodal physiological information maps and a two-stream convolutional hybrid attention fusion network.

[0021] For example, the muscle fatigue detection method based on multimodal physiological information maps proposed in this application is applicable to a muscle fatigue detection device based on multimodal physiological information maps. This muscle fatigue detection device based on multimodal physiological information maps can be deployed on an electronic device. The electronic device can be a computer device configured with a von Neumann architecture, such as a desktop computer, a laptop computer, or a server. The electronic device can also be an electronic device with central control functions, such as a gateway. The electronic device can also be a portable mobile electronic device, such as a smartphone or a tablet computer.

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0023] Please see Figure 1 This application provides a flowchart of a muscle fatigue detection method based on multimodal physiological information maps. This method is applicable to electronic devices, which can be servers or other devices with data analysis and processing capabilities, without specific limitations.

[0024] In the following method embodiments, for ease of description, the execution subject of each step of the method is an electronic device, but this does not constitute a specific limitation.

[0025] like Figure 1 As shown, the method may include the following steps: Step S110: Obtain the spatiotemporal synchronization signal of the target muscle group.

[0026] The spatiotemporal synchronization signal includes electromyographic signals and muscle oxygenation signals.

[0027] Specifically, this application proposes a 60-channel (6×10 matrix) synchronous electromyography (EMG) and muscle oxygenation (MOO) composite sensor array. The sensor array employs an interlaced rhomboid and grid layout, with each rhomboid unit consisting of four EMG electrodes and four MOO probes. The EMG electrodes are located at the midpoint of the line connecting the MOO transmitting and receiving probes, achieving spatial co-localization and coupled acquisition of EMG and MOO modalities within the same area. This application uses the lumbar and back muscles as an example to illustrate the target muscle group. The acquired 6×10 dual-modal matrix data facilitates the subsequent generation of dual-modal physiological information maps.

[0028] The spatiotemporal synchronization signals in this scheme include, but are not limited to: collecting dual-mode signals (electromyography and muscle oxygenation signals) from 11 healthy subjects during 6kg weighted bending and unweighted bending until complete fatigue. Simultaneously, the scheme also collects the Borg Self-Performed Strength Scale (RPE) scores of 6-20 provided by the subjects in real time. Scores of 6-11 indicate low fatigue, 12-16 indicate moderate fatigue, and 17-20 indicate high fatigue, serving as a 3-level fatigue label for detecting the dual-mode signals during the fatigue process. Furthermore, each score from 6-20 is considered a fatigue level, serving as a 15-level fatigue label for detecting the dual-mode signals during the fatigue process.

[0029] Step S120: Perform data preprocessing on the spatiotemporal synchronization signal to extract target feature parameters reflecting muscle fatigue state.

[0030] Specifically, multimodal physiological information maps are the input basis for target detection models. Their core is to transform the original signals of two modes—electromyography, which reflects neuromuscular electrical activity, and muscle oxygenation, which reflects tissue oxygenation status (i.e., spatiotemporal synchronization signals)—into a visualized feature topographic map containing spatial distribution information through data acquisition and signal processing.

[0031] Before obtaining the multimodal physiological information map, the spatiotemporal synchronization signal needs to be preprocessed to extract features and obtain target feature parameters that reflect muscle fatigue state.

[0032] Specifically, data preprocessing includes two steps: denoising the acquired raw signal and feature extraction. The denoising process aims to remove motion artifacts and environmental interference to improve the accuracy of feature extraction.

[0033] Step S130: Visualize the target feature parameters to obtain a multimodal physiological information map.

[0034] The multimodal physiological information map is a topographic map that transforms the discretely arranged spatiotemporal synchronization signals into a visual feature map containing spatial distribution information. The multimodal physiological information map is used to characterize the features of the two modes of electromyography signal and muscle oxygenation signal.

[0035] Specifically, based on the target feature parameters extracted after data preprocessing, multimodal physiological information maps are generated through interpolation and visualization.

[0036] Step S140: Input the multimodal physiological information map as input data into the target detection model to obtain muscle fatigue classification results.

[0037] The target detection model is constructed based on a two-stream convolutional hybrid attention fusion network.

[0038] Specifically, this application proposes a dual-stream convolutional hybrid attention network that integrates electromyography (EMG) and muscle oxygenation multimodal physiological information as a target detection model for detecting fatigue in the lumbar and back muscles. The network uses surface EMG and muscle oxygenation multimodal physiological information maps collected from the lumbar and back areas as input data. It is trained using a supervised learning method on a feature topographic map training set labeled with fatigue levels, achieving objective and accurate classification of lumbar and back muscle fatigue levels.

[0039] Specifically, the dual-stream convolutional hybrid attention network consists of three main modules: electromyography feature extraction stream (processing electromyography topography), muscle oxygenation feature extraction stream (processing muscle oxygenation topography), and hierarchical attention fusion module (fusion of dual-modal features and classification). The three modules work together to achieve accurate feature extraction and further achieve cross-modal information complementarity, ultimately improving the accuracy of fatigue level classification.

[0040] Under the above embodiments, the method proposed in this application first acquires the spatiotemporal synchronization signal of the target muscle group, wherein the spatiotemporal synchronization signal includes electromyographic signals and muscle oxygenation signals; then, the spatiotemporal synchronization signal is preprocessed to extract target feature parameters reflecting muscle fatigue state; further, visualization processing is performed based on the target feature parameters to obtain a multimodal physiological information map; finally, the multimodal physiological information map is used as input data to the target detection model to obtain muscle fatigue classification results. Thus, based on the acquired spatiotemporal synchronization signal and the target detection model, the fusion of the multimodal physiological information map is achieved, thereby completing the identification of muscle fatigue, ultimately improving the objectivity and accuracy of lumbar and back muscle fatigue detection and classification, and providing an objective and effective preventive measure for chronic low back pain and acute muscle injury.

[0041] In an exemplary embodiment, the method for acquiring the spatiotemporal synchronization signal of the target muscle group further includes the following steps: acquiring 60 channels of surface electromyography (EMG) signals of the target muscle group through a transparent, flexible, and stretchable electrode array to obtain the EMG signal, wherein the EMG signal is used to record real-time changes in neuromuscular electrical activity; acquiring the blood oxygen dynamics signal of the target muscle group through 18 transmitting probes and 20 receiving probes to obtain the muscle oxygen signal, wherein the muscle oxygen signal is used to reflect the muscle metabolic state; the EMG signal and the muscle oxygen signal together constitute the spatiotemporal synchronization signal.

[0042] Specifically, in this application, a 60-channel synchronous electromyography (EMG) and muscle oxygenation combined sensor array (6×10 matrix layout) is used to achieve spatial co-location coupling acquisition of the two modalities. The specific array design diagram and an example of generated EMG feature topographic map are shown below. Figure 2 As shown.

[0043] Specifically, the electromyography (EMG) signal was acquired using a 6-row, 10-column transparent, flexible, and stretchable electrode array (21.2 mm spacing between adjacent electrodes) in the EMG acquisition module. This method acquired 60 channels of surface EMG signals from the L2-L5 region of the lower back, recording real-time changes in neuromuscular electrical activity and meeting the spatial resolution requirements for EMG potential difference acquisition. The transparency of the EMG electrodes ensures that light energy from the oxygen emission electrodes can pass through them to achieve dual-source coupling acquisition, while the stretchability of the electrodes ensures good fit against the lower back.

[0044] The muscle oxygenation signal is acquired through a muscle oxygenation acquisition module consisting of 18 transmitting probes and 20 receiving probes, with a 30 mm spacing between the transmitting and receiving probes. This module collects blood oxygenation dynamics signals from the same area and converts them into changes in oxyhemoglobin (ΔHbO) and deoxyhemoglobin (ΔHbR) to reflect muscle metabolic status. This ensures that the muscle oxygenation probe can penetrate the skin and superficial muscles within the optimal range to obtain stable blood oxygenation dynamics information. Furthermore, the muscle oxygenation probe belt is made of flexible and elastic rubber, improving wearing comfort. The entire array covers an area of ​​21.25 cm × 12.75 cm, providing the necessary spatial coverage and data support for subsequent dual-modal physiological topography generation.

[0045] In an exemplary embodiment, the electromyographic signal and the muscle oxygenation signal are synchronized and the signals are acquired for the same region of the target muscle group.

[0046] Specifically, during dual-mode acquisition, six rows of ten flexible, stretchable electromyographic electrodes are first evenly attached to the lumbar muscle region at the spinal level from L2 to L5 of the subject. Two additional electrodes are attached to the surfaces of the sacral vertebrae on both sides, serving as the reference electrode (REF) and ground electrode (GND), respectively. Then, based on the positions of the surface electromyographic electrodes, a muscle oxygen probe belt is placed over the same area to ensure a one-to-one correspondence between the electromyographic electrodes and the muscle oxygen channels.

[0047] Under the above embodiments, by developing a 60-channel synchronous electromyography and muscle oxygenation composite sensor array, it is possible to achieve synchronous coupling acquisition of electromyography and muscle oxygenation signals, providing the necessary spatial coverage and data support for the subsequent generation of multimodal physiological information maps.

[0048] In an exemplary embodiment, the method for data preprocessing of the spatiotemporal synchronization signal includes the following steps: after denoising the spatiotemporal synchronization signal, feature extraction is performed on the electromyography (EMG) signal and the muscle oxygenation signal to obtain the feature parameters. Specifically, the feature extraction includes the following steps: extracting the root mean square (RMS) and median frequency of each segment of the EMG signal to obtain a first feature parameter, wherein the RMS reflects the intensity of the EMG signal and the median frequency reflects the change in EMG frequency; retaining the changes in oxyhemoglobin and deoxyhemoglobin in the muscle oxygenation signal after denoising to obtain a second feature parameter, wherein the change in oxyhemoglobin reflects the balance between oxygen supply and consumption in the muscle, and the change in deoxyhemoglobin shows a negative correlation with the change in oxyhemoglobin, reflecting the oxygen consumption state of the muscle; the first feature parameter and the second feature parameter together constitute the target feature parameter.

[0049] Specifically, electromyographic features: The first feature parameter is obtained by extracting the root mean square (RMS) reflecting the intensity of the electromyographic signal and the median frequency (MDF) reflecting the change in the electromyographic frequency from each segment of the electromyographic signal.

[0050] Specifically, in the muscle oxygenation signal feature extraction process, the core feature parameters are the change in oxyhemoglobin (ΔHbO) and the change in deoxyhemoglobin (ΔHbR). These two parameters are key physiological indicators directly extracted from the muscle oxygenation signal after preprocessing, and also the core data basis for the subsequent generation of muscle oxygenation feature topography maps. Specific details are as follows: Muscle oxygenation signal acquisition relies on near-infrared spectroscopy (NIRS). An array of 18 transmitting probes emits near-infrared light, while 20 receiving probes receive the light intensity signal after penetrating the skin and superficial muscles. Since hemoglobin in different states (oxygenated / deoxygenated) absorbs near-infrared light differently, changes in the concentration of the two types of hemoglobin can be inferred from changes in light intensity, ultimately yielding two core parameters (i.e., the second characteristic parameters): the change in oxygenated hemoglobin (ΔHbO) and the change in deoxygenated hemoglobin (ΔHbR).

[0051] Specifically, ΔHbO refers to the change in the concentration of oxygenated hemoglobin (HbO2) in the back muscles relative to the initial state during the testing process (usually expressed in μmol / L). ΔHbO reflects the balance between oxygen supply and consumption in muscles. When muscles are not fatigued, oxygen supply can meet consumption, and ΔHbO remains relatively stable. As fatigue intensifies, muscle oxygen consumption increases or oxygen supply becomes insufficient, and ΔHbO usually shows a decreasing trend (this needs to be judged in conjunction with the specific exercise scenario; for example, it may briefly rise and then fall at the beginning of high-intensity exercise).

[0052] Specifically, ΔHbR refers to the change in the concentration of deoxyhemoglobin (HbR) in the back muscle tissue relative to its initial state during the testing process (units are the same as ΔHbO). ΔHbR reflects the oxygen consumption status of muscles and is negatively correlated with ΔHbO. When muscle activity increases (fatigue accumulates), hemoglobin releases oxygen to provide energy, deoxyhemoglobin increases, and ΔHbR usually shows an upward trend.

[0053] Furthermore, through noise reduction processing in the preprocessing operation, interference information such as motion artifacts (such as probe displacement caused by back muscle activity) and ambient light interference that are easily affected by muscle oxygenation signals is removed, and abnormal signal fluctuations caused by motion are separated and eliminated to ensure that ΔHbO and ΔHbR can truly reflect blood oxygenation changes related to muscle fatigue.

[0054] Furthermore, the preprocessed ΔHbO and ΔHbR signals are segmented according to a fixed time length (the segment length matches the segmentation of the electromyographic signal, such as every 2-5 seconds). Each segment corresponds to a muscle oxygenation feature parameter within a "time window," and the signal segmentation is consistent with the time dimension of electromyographic feature extraction (to facilitate subsequent multimodal fusion). The first and second feature parameters together constitute the target feature parameters required to generate a multimodal physiological signal map.

[0055] Under the above embodiments, the initial signal data (i.e., the spatiotemporal synchronization signal) is denoised through data preprocessing, and the core feature parameters representing the two modes are extracted to obtain the final target feature parameters for subsequent construction of multimodal physiological information maps. This is beneficial for more accurately revealing the spatial coordination and temporal complementarity information between modes, improving the key accuracy of multimodal physiological information maps, and thus improving the accuracy of fatigue classification results.

[0056] In an exemplary embodiment, the visualization processing based on the target feature parameters includes the following steps: filling feature gaps using cubic spline interpolation based on the actual positions of the 60 channels in the target muscle group to form a continuous spatial feature matrix, wherein each element in the spatial feature matrix corresponds to a feature value; mapping each feature value to a color intensity to obtain the multimodal physiological information map, wherein the multimodal physiological information map includes an electromyographic topography map and a muscle oxygenation topography map, and the multimodal physiological information map visually presents the spatial distribution differences of the two modal features in the manner of the visualized feature topography map.

[0057] Specifically, after extracting the target feature parameters, electromyography topography and muscle oxygenation topography are generated using the target feature parameters, and finally a multimodal physiological information map is obtained.

[0058] Specifically, the extracted second feature parameters ΔHbO and ΔHbR are not directly input into the fusion network, but need to be further transformed into muscle oxygenation topography (i.e., spatial representation of the second feature parameters). The specific process is as follows: spatial interpolation is used to complete the segmented data of ΔHbO and ΔHbR collected by the 60-channel sensor array. Cubic spline interpolation is used to fill the feature gaps between channels according to the actual spatial location of each channel in the lower back (covering the L2-L5 lumbar muscle region) to form a continuous spatial feature matrix. The matrix dimension matches the spatial layout of the sensor array, for example, corresponding to a 21.25cm × 12.75cm coverage area.

[0059] Furthermore, the concentration variation value of ΔHbO (or ΔHbR) at each pixel in the spatial feature matrix is ​​mapped to the color intensity of the RGB three primary colors through visualization mapping. For example, the greater the concentration variation, the darker or brighter the color. The specific mapping rules can be determined according to the data range and are not specifically limited here. Finally, two types of muscle oxygenation feature topographic maps are generated: △HbO characteristic topographic map: visually presents the spatial distribution and variation trend of oxygenated hemoglobin in different areas of the lower back; △HbR characteristic topographic map: visually presents the spatial distribution and variation trend of deoxygenated hemoglobin in different areas of the lower back.

[0060] Under the above embodiments, the two types of topographic maps obtained based on the target feature parameters will be used as inputs to the muscle oxygenation feature extraction stream, and will be jointly entered into the subsequent dual-stream convolutional hybrid attention fusion network with the electromyography topographic map, providing spatial feature support for muscle oxygenation mode for fatigue level classification.

[0061] In an exemplary embodiment, the method of inputting the multimodal physiological information map as input data into a target detection model to obtain a muscle fatigue classification result includes: extracting key features from the electromyography topography map and the muscle oxygenation topography map respectively to obtain electromyography features and muscle oxygenation features; performing feature fusion based on the electromyography features and the muscle oxygenation features to obtain fused features containing global cross-modal associations; and outputting a fatigue level based on the fused features to obtain the muscle fatigue classification result.

[0062] Specifically, the target detection model (i.e., the dual-stream convolutional hybrid attention fusion network) in this application consists of an electromyography feature extraction stream designed based on the characteristics of electromyography feature topography, a muscle oxygenation feature extraction stream designed based on the characteristics of muscle oxygenation topography, and a hierarchical attention fusion module.

[0063] For example, please refer to Figure 3 The diagram shown illustrates the fatigue classification process of the target detection model, as follows: Figure 3 The target detection model shown is a two-stream convolutional hybrid attention fusion network, where the convolutional layer (7,32) refers to a convolutional kernel size of 7×7 and 32 output channels. Specifically, the two-stream convolutional hybrid attention fusion network consists of an electromyography feature extraction stream designed based on the characteristics of electromyography feature topography, a muscle oxygen feature extraction stream designed based on the characteristics of muscle oxygen feature topography, and a hierarchical attention fusion module.

[0064] For example, the electromyography (EMG) and oxygenation (OO) feature extraction streams are specifically constructed by concatenating the EMG topographic maps (MDF, RMS) and OO topographic maps (ΔHbO, ΔHbR) by channel, and then inputting them into the EMG and OO feature extraction streams respectively. Each EMG and OO feature extraction stream consists of three sequential convolutional blocks, one parallel multi-scale convolutional block, and one spatial-channel attention module.

[0065] Specifically, the three sequential convolutional blocks in the electromyography (EMG) and muscle oxygenation branches are identical. Each convolutional block consists of a convolutional layer (Conv), a batch normalization (BN) layer, an activation function (ReLU), and a max pooling layer. The three convolutional layers use decreasing kernel sizes of 7×7, 5×5, and 3×3 to achieve layer-by-layer information capture from global to local. BN and ReLU can alleviate gradient vanishing and suppress overfitting, improving the network's generalization performance. The max pooling layer has a kernel size and stride of 2.

[0066] The multi-scale convolutional block comprises three parallel branches. In the electromyography (EMG) feature extraction stream, due to the drastic changes in EMG signals, adaptive max pooling is used in the first branch to capture rapid changes in muscle activity. The second branch uses 1×1 convolutions to achieve cross-channel information exchange, followed by BN, ReLU, and Maxpooling. The third branch is similar to the second, but its convolutional layers use smaller 3×3 kernels. Finally, the outputs of the three parallel branches are concatenated to form a fused feature map. The multi-scale convolutional block in the muscle oxygenation feature extraction stream is similar to that of EMG, but because muscle oxygenation reflects changes in muscle blood oxygenation, which have a more gradual trend, adaptive average pooling is used in the first branch, a larger 5×5 kernel in the third branch, and the max pooling layers in the second and third branches are replaced with average pooling layers.

[0067] The basis for the differentiated design of multi-scale convolutional blocks is as follows: Electromyographic signal characteristics: Reflects neuromuscular electrophysiological activity. During muscle fatigue, the changes are dramatic and highly localized (e.g., rapid fluctuations in potential during muscle activity). Therefore, the multi-scale convolutional block of the electromyographic feature extraction flow is as follows: Branch 1 uses adaptive max pooling to capture rapid peak changes in muscle activity; Branch 3 uses 3×3 small convolutional kernels to focus on extracting local high-frequency information.

[0068] Muscle oxygenation signal characteristics: It reflects the oxygenation status of muscle tissue, with a gradual trend and a wide spatial correlation (e.g., blood oxygen concentration changes slowly with metabolism and has a wide range of influence). Therefore, the multi-scale convolutional block of the muscle oxygenation feature extraction flow is as follows: Branch 1 uses adaptive average pooling to smooth the smooth signal and avoid peak interference; Branch 3 uses a 5×5 large convolutional kernel to capture a wider range of spatial oxygenation information; at the same time, the max pooling in Branches 2 and 3 is changed to average pooling to further adapt to the feature extraction requirements of the smooth signal.

[0069] For example, the spatial-channel attention module consists of spatial attention and channel attention, focusing on important spatial and channel features of electromyography and muscle oxygenation. It takes the feature map extracted by convolution as input, and the specific calculation process is as follows: Where Fs and Fc are the spatial attention map and the channel attention map, respectively. For a 2D convolution operation with a kernel size of l×l, It is the ReLU activation function. For the sigmoid function, This is an element-wise multiplication. In spatial attention, to capture rapid local changes in electromyographic signals, the electromyographic feature extraction stream... =3; In order to capture a wider range of spatial information in the muscle oxygenation signal, the muscle oxygenation feature extraction stream... =7.

[0070] For example, after the electromyography (EMG) feature extraction stream and the muscle oxygenation (MO) feature extraction stream have completed feature extraction, in order to effectively fuse the features of EMG and MO, this study designed a hierarchical attention fusion module based on multi-head attention.

[0071] For the feature maps output from the EMG and OX feature extraction streams, they are first flattened into sequences in the spatial dimension. Then, a two-dimensional sine and cosine function is used to generate a position embedding for each spatial location based on the spatial dimension of the original feature map, and this embedding is added to the corresponding feature vector to inject spatial position information. In the first step of fusion, the EMG sequence with position encoding and the OX sequence are concatenated and input into a joint self-attention layer for joint modeling. With the help of multi-head self-attention, cross-modal features are weighted and fused globally to obtain a globally fused feature that preserves the instantaneous changes in EMG and the slow changes in OX. The calculation process is as follows: in, It is a combined sequence composed of position-coded electromyographic and oximetric sequences. . This represents multi-head self-attention. LN represents layer normalization, and MLP represents multilayer perceptron.

[0072] Furthermore, after obtaining the global fusion information, the different modalities are further fused in the cross-attention layer. The electromyography cross-attention layer operates as follows: in, It is multi-head cross-attention, and the query (Q) of MHCA in the electromyography cross-attention layer is: The key (K) and value (V) are The difference between the muscle oxygenation cross-attention layer and the electromyography cross-attention layer is that the query (Q) is... .

[0073] Finally, the outputs of the electromyography cross-attention layer and the muscle oxygenation cross-attention layer are spliced ​​and sent to the fully connected layer (FC) for classification, and the corresponding muscle fatigue level is output to obtain the final fatigue classification result.

[0074] Furthermore, the model was validated using five-fold cross-validation, and the Adam optimizer was used to optimize the cross-entropy loss function. Hyperparameters were selected by minimizing the validation loss. The initial learning rate of the model was 0.0001, the number of training epochs was 30, and the batch size was 8.

[0075] The accuracy of the dual-stream convolutional attention network in detecting level 3 fatigue and level 15 fatigue on 11 subjects is as follows: Figure 4 As shown in the figure. Experimental results demonstrate that the two-stream convolutional attention network exhibits satisfactory performance in fatigue recognition. The confusion matrix for fatigue recognition of Subject 3 is shown in the figure. Figure 5 As shown.

[0076] Furthermore, this application's solution will also integrate the results of sEMG and NIRS with those obtained from sEMG or NIRS alone at fatigue levels 3 and 15, such as... Figure 5 As shown, the evaluation metrics include accuracy, precision, recall, and F1 score. The fatigue identification results achieved through fusion are superior to those achieved through individual modalities. Repeated measures ANOVA shows significant differences among the three modalities. When significance is found in the repeated measures ANOVA results, the Least Significant Difference (LSD) method is used for pairwise post-hoc comparisons. The results show that the fusion scheme significantly outperforms the single-modal scheme in all metrics. This demonstrates the statistical reliability of the fusion advantage.

[0077] Confusion matrix verification: The confusion matrix of the three-level fatigue detection for some subjects (such as subject 3) shows that the classification accuracy of low, medium and high fatigue reached 98.46%, 95.76% and 98.08% respectively. The 15-level detection also showed a low error rate, which intuitively shows that the fusion module improved the classification accuracy and was better than the coarse results of the existing weighted fusion.

[0078] The above embodiments and experimental results demonstrate that the target detection model designed in this application, a dual-stream convolutional hybrid attention network, fully utilizes multimodal complementary information by fusing feature extraction streams adapted to modal characteristics with hierarchical attention, ultimately improving detection accuracy and generalization ability. Furthermore, the detection results are objective and accurate, requiring no subjective evaluation, and achieve objective classification of lumbar and back muscle fatigue, providing a reliable means for injury prevention.

[0079] In summary, the proposed solution has undergone thorough experiments and comparative verification. The model demonstrates the superiority of the proposed solution in "fatigue detection classification performance" and possesses good practicality, stability, and prospects for promotion, thus having significant biomedical value.

[0080] It should be noted that, for ease of intuitive understanding, the muscle fatigue detection method based on multimodal physiological information maps in this application will be explained in detail with the detection of muscle fatigue in the lower back as an example. In practical applications, the detection of muscle fatigue in other parts of the body can be adapted and adjusted according to specific needs, and no specific limitations are made here.

[0081] The following are embodiments of the apparatus described in this application, which can be used to execute the muscle fatigue detection method based on multimodal physiological information maps involved in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the method embodiments of the muscle fatigue detection method based on multimodal physiological information maps involved in this application.

[0082] Please see Figure 6 This application provides a muscle fatigue detection device 600 based on multimodal physiological information maps. The device includes a signal acquisition module 601, a preprocessing module 602, a visualization processing module 603, and a result output module 604, wherein: The signal acquisition module 601 is used to acquire the spatiotemporal synchronization signal of the target muscle group, wherein the spatiotemporal synchronization signal includes electromyographic signal and muscle oxygenation signal; The preprocessing module 602 is used to preprocess the spatiotemporal synchronization signal to extract target feature parameters that reflect muscle fatigue state. The visualization processing module 603 is used to perform visualization processing based on the target feature parameters to obtain a multimodal physiological information map. The multimodal physiological information map is obtained by converting the discretely arranged spatiotemporal synchronization signals into a visualized feature topographic map containing spatial distribution information. The multimodal physiological information map is used to characterize the features of the two modes of electromyography signal and muscle oxygenation signal. The result output module 604 is used to input the multimodal physiological information map as input data into the target detection model to obtain muscle fatigue classification results. The target detection model is constructed based on a two-stream convolutional hybrid attention fusion network.

[0083] Therefore, in this scheme, based on the aforementioned muscle fatigue detection device based on multimodal physiological information maps, firstly, the signal acquisition module acquires the spatiotemporal synchronization signal of the target muscle group, which includes electromyographic (EMG) signals and muscle oxygenation signals; then, the preprocessing module performs data preprocessing on the spatiotemporal synchronization signal to extract target feature parameters reflecting the muscle fatigue state; further, the visualization processing module performs visualization processing based on the target feature parameters to obtain the multimodal physiological information map; finally, the result output module inputs the multimodal physiological information map as input data into the target detection model to obtain the muscle fatigue classification result. In this way, the fusion of multimodal physiological information maps is achieved based on the acquired spatiotemporal synchronization signal and the target detection model, thereby completing the identification of muscle fatigue and ultimately improving the objectivity and accuracy of lumbar and back muscle fatigue detection and classification, providing an objective and effective preventive measure for chronic low back pain and acute muscle injury.

[0084] In an exemplary embodiment, the signal acquisition module 601 is further configured to: acquire the spatiotemporal synchronization signal of the target muscle group. The electromyographic signals are obtained by acquiring 60 channels of surface electromyographic signals of the target muscle group through a transparent, flexible, and stretchable electrode array. The electromyographic signals are used to record real-time changes in neuromuscular electrical activity. The muscle oxygen signal is obtained by acquiring the blood oxygen dynamics signal of the target muscle group through 18 transmitting probes and 20 receiving probes, wherein the muscle oxygen signal is used to reflect the muscle metabolic state.

[0085] In an exemplary embodiment, the signal acquisition module 601 is further configured to: acquire the electromyographic signal and the muscle oxygenation signal synchronously.

[0086] In an exemplary embodiment, the preprocessing module 602 further comprises performing data preprocessing on the spatiotemporal synchronization signal, wherein the preprocessing module 602 is configured to: After denoising the spatiotemporal synchronization signal, feature extraction is performed on the electromyography signal and the muscle oxygenation signal to obtain the feature parameters. The feature extraction specifically includes the following steps: The root mean square (RMS) and median frequency of each segment of the electromyography (EMG) signal are extracted to obtain a first feature parameter, wherein the RMS reflects the intensity of the EMG signal and the median frequency reflects the change in EMG frequency. The changes in oxyhemoglobin and deoxyhemoglobin after the denoising process of the muscle oxygen signal are retained to obtain a second characteristic parameter. The change in oxyhemoglobin is used to reflect the balance between oxygen supply and oxygen consumption in the muscle, and the change in deoxyhemoglobin is negatively correlated with the change in oxyhemoglobin. The change in deoxyhemoglobin is used to reflect the oxygen consumption status of the muscle. The first feature parameter and the second feature parameter together constitute the target feature parameter.

[0087] In an exemplary embodiment, the visualization processing module 603, which performs visualization processing based on the target feature parameters, is further configured to: Based on the actual location of the 60 channels in the target muscle group, cubic spline interpolation is used to fill the feature gaps to form a continuous spatial feature matrix, wherein each element in the spatial feature matrix corresponds to a feature value; Each feature value is mapped to a color intensity to obtain the multimodal physiological information map, wherein the multimodal physiological information map includes an electromyographic topographic map and a muscle oxygenation topographic map, and the multimodal physiological information map intuitively presents the spatial distribution differences of the two modal features in the form of the visualized feature topographic map.

[0088] In an exemplary embodiment, the step of inputting the multimodal physiological information map as input data into the target detection model to obtain muscle fatigue classification results, wherein the result output module 604 is further configured to: Key features of the electromyography topography map and the muscle oxygenation topography map are extracted respectively to obtain electromyography features and muscle oxygenation features; Based on the electromyographic features and the muscle oxygenation features, feature fusion is performed to obtain fused features that include global cross-modal correlations; The fatigue level is output based on the fusion features to obtain the muscle fatigue classification result.

[0089] It should be noted that the muscle fatigue detection device based on multimodal physiological information maps provided in the above embodiments is only illustrated by the division of the above functional modules when performing gene detection and analysis. In actual applications, the above functions can be assigned to different functional modules as needed. That is, the internal structure of the muscle fatigue detection device based on multimodal physiological information maps will be divided into different functional modules to complete all or part of the functions described above.

[0090] Furthermore, the muscle fatigue detection device based on multimodal physiological information maps and the muscle fatigue detection method based on multimodal physiological information maps provided in the above embodiments belong to the same concept. The specific way each module performs its operation has been described in detail in the method embodiments, and will not be repeated here.

[0091] It should be noted that the description of the above device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0092] This application also provides an electronic device. Figure 7 This is a schematic diagram of an optional structure of an electronic device provided in an embodiment of this application. For example... Figure 7 As shown, the electronic device 7000 includes at least one processor 7001 and at least one memory 7003.

[0093] The data interaction between the processor 7001 and the memory 7003 can be achieved through at least one communication bus 7002. This communication bus 7002 may include a path for transmitting data between the processor 7001 and the memory 7003. The communication bus 7002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The communication bus 7002 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0094] Optionally, the electronic device 7000 may further include a transceiver 7004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 7004 is not limited to one type, and the structure of the electronic device 7000 does not constitute a limitation on the embodiments of this application.

[0095] Processor 7001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 7001 may also be a combination that implements computing functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0096] The memory 7003 may be a ROM (Read Only Memory) or other type of static storage device capable of storing static information and instructions, RAM (Random Access Memory) or other type of dynamic storage device capable of storing information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program instructions or code in the form of instructions or data structures and accessible by the electronic device 7000, but not limited thereto.

[0097] The memory 7003 stores program instructions or code, and the processor 7001 can read the program instructions or code stored in the memory 7003 through the communication bus 7002.

[0098] When the program instructions or code are executed by the processor 7001, the muscle fatigue detection method based on multimodal physiological information maps in the above embodiments is implemented.

[0099] This application provides a computer-readable storage medium storing executable instructions. When the executable instructions are executed by the processor, the processor will execute the muscle fatigue detection method based on multimodal physiological information maps provided in this application.

[0100] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0101] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0102] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts within a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files storing one or more modules, subroutines, or code sections). As an example, executable instructions may be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0103] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0104] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0105] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0106] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0107] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

[0108] Compared with related technologies, the muscle fatigue detection method based on multimodal physiological information graphs proposed in this application involves: acquiring the text to be reviewed; performing a peer review simulation on the text to obtain simulation results, which include at least one reviewer comment and at least one author response; performing structured processing on the simulation results to obtain a heterogeneous debate graph, which represents the relationship between reviewer comments and author responses; and performing reasoning operations based on the heterogeneous debate graph to obtain the review decision result. In this way, by simulating multiple rounds of debate between reviewers and authors, and performing structured analysis of the simulation results to construct a heterogeneous debate graph for semantic reasoning, a more structured, stable, and interpretable automated peer review decision is achieved.

[0109] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0110] The above description is only a partial embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for detecting muscle fatigue based on multimodal physiological information maps, characterized in that, include: Acquire spatiotemporal synchronization signals of the target muscle group, wherein the spatiotemporal synchronization signals include electromyographic signals and muscle oxygenation signals; The spatiotemporal synchronization signal is preprocessed to extract target feature parameters reflecting muscle fatigue state; Based on the target feature parameters, visualization processing is performed to obtain a multimodal physiological information map. The multimodal physiological information map is obtained by transforming the discretely arranged spatiotemporal synchronization signals into a visualized feature topographic map containing spatial distribution information. The multimodal physiological information map is used to characterize the features of the two modes of electromyography signal and muscle oxygenation signal. The multimodal physiological information map is used as input data to the target detection model to obtain muscle fatigue classification results. The target detection model is constructed based on a two-stream convolutional hybrid attention fusion network.

2. The method as described in claim 1, characterized in that, The method for acquiring the spatiotemporal synchronization signal of the target muscle group includes: The electromyographic signals are obtained by acquiring 60 channels of surface electromyographic signals of the target muscle group through a transparent, flexible, and stretchable electrode array. The electromyographic signals are used to record real-time changes in neuromuscular electrical activity. The muscle oxygen signal is obtained by collecting blood oxygen dynamics signals of the target muscle group through 18 transmitting probes and 20 receiving probes, wherein the muscle oxygen signal is used to reflect the muscle metabolic state. The electromyographic signal and the muscle oxygenation signal together constitute the spatiotemporal synchronization signal.

3. The method as described in claim 2, characterized in that, The electromyographic signal and the muscle oxygenation signal are synchronized and the signals are collected from the same area of ​​the target muscle group.

4. The method as described in claim 1, characterized in that, The method for preprocessing the spatiotemporal synchronization signal includes: After denoising the spatiotemporal synchronization signal, feature extraction is performed on the electromyography signal and the muscle oxygenation signal to obtain the feature parameters. The feature extraction specifically includes the following steps: The root mean square (RMS) and median frequency of each segment of the electromyography (EMG) signal are extracted to obtain a first feature parameter, wherein the RMS reflects the intensity of the EMG signal and the median frequency reflects the change in EMG frequency. The changes in oxyhemoglobin and deoxyhemoglobin after the denoising process of the muscle oxygen signal are retained to obtain a second characteristic parameter. The change in oxyhemoglobin is used to reflect the balance between oxygen supply and oxygen consumption in the muscle, and the change in deoxyhemoglobin is negatively correlated with the change in oxyhemoglobin. The change in deoxyhemoglobin is used to reflect the oxygen consumption status of the muscle. The first feature parameter and the second feature parameter together constitute the target feature parameter.

5. The method as described in claim 3, characterized in that, The method for visualizing based on the target feature parameters includes: Based on the actual location of the 60 channels in the target muscle group, cubic spline interpolation is used to fill the feature gaps to form a continuous spatial feature matrix, wherein each element in the spatial feature matrix corresponds to a feature value; Each feature value is mapped to a color intensity to obtain the multimodal physiological information map, wherein the multimodal physiological information map includes an electromyographic topography map and a muscle oxygenation topography map, and the multimodal physiological information map intuitively presents the spatial distribution differences of the two modal features in the form of the visualized feature topography map.

6. The method as described in claim 5, characterized in that, The method of inputting the multimodal physiological information map as input data into a target detection model to obtain muscle fatigue classification results includes: Key features of the electromyography topography map and the muscle oxygenation topography map are extracted respectively to obtain electromyography features and muscle oxygenation features; Based on the electromyographic features and the muscle oxygenation features, feature fusion is performed to obtain fused features that include global cross-modal correlations; The fatigue level is output based on the fusion features to obtain the muscle fatigue classification result.

7. A muscle fatigue detection device based on multimodal physiological information maps, characterized in that, The device includes a signal acquisition module, a preprocessing module, a visualization processing module, and a result output module, wherein: The signal acquisition module is used to acquire the spatiotemporal synchronization signal of the target muscle group, wherein the spatiotemporal synchronization signal includes electromyographic signal and muscle oxygenation signal; The preprocessing module is used to preprocess the spatiotemporal synchronization signal to extract target feature parameters that reflect muscle fatigue state. The visualization processing module is used to perform visualization processing based on the target feature parameters to obtain a multimodal physiological information map. The multimodal physiological information map is obtained by converting the discretely arranged spatiotemporal synchronization signals into a visualized feature topographic map containing spatial distribution information. The multimodal physiological information map is used to characterize the features of the two modes of electromyography signal and muscle oxygenation signal. The result output module is used to input the multimodal physiological information map as input data into the target detection model to obtain muscle fatigue classification results. The target detection model is constructed based on a two-stream convolutional hybrid attention fusion network.

8. An electronic device, characterized in that, include: At least one processor and at least one memory, wherein, The memory stores program instructions or code; The program instructions or code are loaded and executed by the processor, causing the electronic device to implement the muscle fatigue detection method based on multimodal physiological information maps as described in any one of claims 1 to 6.

9. A storage medium storing program instructions or code thereon, characterized in that, The program instructions or code are loaded and executed by the processor to implement the muscle fatigue detection method based on multimodal physiological information maps as described in any one of claims 1 to 6.

10. A computer program product comprising a computer program that is read and executed by a processor of a computer device, causing the computer device to perform the muscle fatigue detection method based on multimodal physiological information maps as described in any one of claims 1 to 6.

Images