Pelvic floor rehabilitation training state detection method and system based on multi-modal data fusion

Abstract

The invention relates to the technical field of pelvic floor rehabilitation detection, and discloses a pelvic floor rehabilitation training state detection method and system based on multi-modal data fusion. The method comprises the following steps: acquiring multi-modal data of a to-be-rehabilitated user acquired in a rehabilitation training state, wherein the multi-modal data comprises motion modal data and physiological modal data; extracting key information of the multi-modal data to obtain key modal feature data of each modal data in the multi-modal data; performing data fusion on the key modal feature data of each modal data to obtain fused multi-modal data; and inputting the fused multi-modal data into a rehabilitation detection calculation model to obtain a rehabilitation detection result of the to-be-rehabilitated user. According to the method, through multi-modal data fusion, the physiological modal data and the motion modal data are comprehensively utilized, the rehabilitation state of the patient can be evaluated more comprehensively, and therefore the detection accuracy is improved.

Description

Technical Field

[0001] The present application relates to the technical field of pelvic floor rehabilitation detection, and specifically to a pelvic floor rehabilitation training status detection method and system based on multimodal data fusion. Background Art

[0002] Pelvic floor dysfunction refers to various clinical symptoms caused by dysfunction of the pelvic floor muscles, such as urinary incontinence, dysuria, constipation, and pelvic organ prolapse. These problems not only affect patients' physical health but also severely impact their psychological and social lives. Therefore, the research and application of pelvic floor rehabilitation therapy is particularly important.

[0003] Currently, pelvic floor rehabilitation therapy primarily uses physical therapy, behavioral therapy, and biofeedback to help patients restore pelvic floor muscle function. Studies have shown that early intervention and systematic rehabilitation training can significantly improve patients' symptoms and enhance their quality of life. However, traditional pelvic floor rehabilitation training often relies on patients' self-perception and subjective feedback, lacking scientific monitoring methods and real-time data monitoring and feedback mechanisms. This leads to large individual variability in training effects, making it difficult to evaluate training results and achieve precise rehabilitation goals.

[0004] In view of this, there is an urgent need for a pelvic floor rehabilitation training status detection method based on multimodal data fusion that can solve the above problems. Summary of the Invention

[0005] In order to solve the above technical problems, the purpose of the embodiments of the present application is to provide a pelvic floor rehabilitation training status detection method based on multimodal data fusion, a processor and a machine-readable storage medium.

[0006] In order to achieve the above objectives, the first aspect of the present application provides a pelvic floor rehabilitation training status detection method based on multimodal data fusion, comprising:

[0007] Acquiring multimodal data of the user to be rehabilitated collected during rehabilitation training, wherein the multimodal data includes motion modal data and physiological modal data;

[0008] Extracting key information from the multimodal data to obtain key modal feature data of each modal data in the multimodal data;

[0009] The key modal feature data of each modal data is fused to obtain fused multimodal data;

[0010] The fused multimodal data is input into the rehabilitation detection calculation model to obtain the rehabilitation detection results of the user to be rehabilitated.

[0011] Preferably, extracting key information of the multimodal data to obtain key modal feature data of each modal data in the multimodal data includes:

[0012] obtaining an electromyographic signal from the physiological modality data;

[0013] performing a preset process on the electromyographic signal to extract a characteristic frequency therefrom;

[0014] Performing a preset process on the amplitude value of the electromyography signal, and determining the amplitude value of the electromyography signal after the preset process as a muscle contraction strength index value;

[0015] The characteristic frequency and the muscle contraction strength index value are determined as key modal feature data of the physiological modal data.

[0016] Preferably, extracting key information of the multimodal data to obtain key modal feature data of each modal data in the multimodal data includes:

[0017] Extracting speed data and acceleration data of the user to be rehabilitated from the motion modality data;

[0018] Calculating a stability assessment value based on the rate of change of the acceleration data;

[0019] The velocity data, the acceleration data, and the stability evaluation value are determined as key modal feature data of the motion modal data.

[0020] Preferably, before extracting the key information of the multimodal data, the method further includes:

[0021] The physiological modality data is preprocessed using formula (1): Formula (1),

[0022] in, represents the preprocessed physiological modality data, represents the initial physiological modality data, Represents the initial physiological modality data The mean of Represents the initial physiological modality data The cutoff frequency, Represents the initial physiological modality data The average frequency of Represents the initial physiological modality data The standard deviation of .

[0023] Preferably, before extracting the key information of the multimodal data, the method further includes:

[0024] The motion modal data is preprocessed using formula (2): Formula (2),

[0025] in, represents the preprocessed motion modal data, represents the initial motion modal data, Represents the initial motion modal data The mean of Represents the initial motion modal data The cutoff frequency, Represents the initial motion modal data The average frequency of Represents the initial motion modal data The standard deviation of Represents the initial motion modal data The Kalman gain.

[0026] Preferably, the key modal feature data of each modal data is fused to obtain fused multimodal data, including:

[0027] Processing the key modal feature data so that the key modal feature data of the physiological modal data and the key modal data of the motion modal data meet data fusion requirements;

[0028] The key modal data of the physiological modal data and the key modal data of the motion modal data are fused according to formula (3): Formula (3),

[0029] in, Represents the fused multimodal data, which combines the information of physiological modality data and motion modality data. represents the preprocessed physiological modality data, represents the preprocessed motion modal data, Represents preprocessed physiological modality data The weight adjustment factor, Represents the preprocessed motion modal data The weight adjustment factor, represents the Laplace distribution density function, j represents the preprocessed motion modal data The scale factor is used to control the data expansion and contraction in the data fusion process. a represents the preprocessed physiological modality data. The penalty factor, i represents the pre-processed motion modal data The penalty factor, t represents the initial physiological modality data and initial motion modal data The data collection time point, k represents the pre-processed physiological modality data The scaling factor of .

[0030] Preferably, after fusing the key modal feature data of each modal data to obtain fused multimodal data, the method further includes:

[0031] Formula (4) is used to perform data enhancement processing on the fused multimodal data: Formula (4),

[0032] in, represents the fused multimodal data, represents the enhanced fusion data, Represents the fused multimodal data frequency, Represents the fused multimodal data The phase offset, Represents the fused multimodal data The variance of Represents the fused multimodal data The mean of .

[0033] Preferably, the fused multimodal data is input into a rehabilitation test calculation model to obtain rehabilitation test results for the user to be rehabilitated, including:

[0034] The rehabilitation detection calculation model uses formula (5) to calculate the multimodal data to obtain the rehabilitation detection results of the user to be rehabilitated: Formula (5),

[0035] Among them, C represents the rehabilitation status detection index, which represents the patient's rehabilitation status score. represents the enhanced fused multimodal data, represents the preprocessed physiological modality data, represents the preprocessed motion modal data, Represents the fused multimodal data The impact factor, Represents preprocessed physiological modality data and preprocessed motion modal data The attenuation factor, Indicates the recovery status threshold.

[0036] The second aspect of the present application provides a pelvic floor rehabilitation training status detection system based on multimodal data fusion, comprising:

[0037] a memory configured to store instructions; and

[0038] The processor is configured to call instructions from the memory and implement the above-mentioned pelvic floor rehabilitation training status detection method based on multimodal data fusion when executing the instructions.

[0039] A third aspect of the present application provides a machine-readable storage medium having instructions stored thereon, the instructions being used to enable a machine to execute the above-mentioned pelvic floor rehabilitation training status detection method based on multimodal data fusion.

[0040] Through the above technical solution, multimodal data of the user to be rehabilitated collected during rehabilitation training is obtained, wherein the multimodal data includes motion modal data and physiological modal data; key information of the multimodal data is extracted to obtain key modal feature data of each modal data in the multimodal data; the key modal feature data of each modal data is fused to obtain fused multimodal data; the fused multimodal data is input into the rehabilitation detection calculation model to obtain the rehabilitation detection results of the user to be rehabilitated. The method of the present invention can more comprehensively evaluate the patient's rehabilitation status through multimodal data fusion and comprehensive utilization of physiological modal data and motion modal data, thereby improving the accuracy of detection; the method of the present invention can monitor the state changes during rehabilitation training and provide timely feedback to the patient to help him adjust the training plan and ensure the effectiveness of rehabilitation training.

[0041] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the following detailed description, they are used to explain the embodiments of the present application but do not constitute a limitation on the embodiments of the present application. In the accompanying drawings:

[0043] Figure 1 A flowchart of a pelvic floor rehabilitation training status detection method based on multimodal data fusion provided in an embodiment of the present application;

[0044] Figure 2 A flowchart of a method for training a rehabilitation detection calculation model in a pelvic floor rehabilitation training state detection method based on multimodal data fusion provided in an embodiment of the present application;

[0045] Figure 3 This is a structural block diagram of a pelvic floor rehabilitation training status detection system based on multimodal data fusion provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] To make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the specific implementation methods described herein are only used to illustrate and explain the embodiments of the present application and are not used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0047] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back, etc.), such directional indications are only used to explain the relative position relationship, movement status, etc. between the various components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0048] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of such features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist, and is not in the flow chart of the pelvic floor rehabilitation training state detection method based on multimodal data fusion required by this application.

[0049] like Figure 1 As shown, the pelvic floor rehabilitation training status detection method based on multimodal data fusion according to an embodiment of the present invention may include the following steps:

[0050] Step 10: Acquire multimodal data of the user to be rehabilitated collected during rehabilitation training, wherein the multimodal data includes motion modal data and physiological modal data;

[0051] Step 20: extract key information of the multimodal data to obtain key modal feature data of each modal data in the multimodal data;

[0052] Step 30: fusing the key modal feature data of each modal data to obtain fused multimodal data;

[0053] Step 40: Input the fused multimodal data into a rehabilitation test calculation model to obtain rehabilitation test results for the user to be rehabilitated.

[0054] Traditional diagnostic methods for pelvic floor dysfunction often focus on only a single indicator, making it difficult to comprehensively assess the functional status of the pelvic floor muscles and having significant limitations.

[0055] The Rise of Multimodal Data Fusion: With the advancement of sensor technology, data processing technology, and artificial intelligence, multimodal data fusion technology has gradually been introduced into the healthcare field. Multimodal data fusion refers to the integration of data from different sources to obtain more comprehensive and accurate information. In the field of pelvic floor rehabilitation, the use of various sensors, such as bioelectric signal sensors, pressure sensors, and motion sensors, to collect data allows real-time monitoring of patients' training status, providing a scientific basis for rehabilitation training. The application of intelligent algorithms, based on data fusion, provides new possibilities for status monitoring of pelvic floor rehabilitation training. Using technologies such as machine learning and deep learning, collected multimodal data can be analyzed to identify the user's training pattern, intensity, and effectiveness. This intelligent monitoring not only improves the scientific nature and accuracy of training but also provides patients with personalized rehabilitation advice.

[0056] In an embodiment of the present invention, during pelvic floor rehabilitation training, multimodal data must first be collected using wearable devices and cameras. Pelvic floor electromyographic signals are important for determining the functional status of a patient's pelvic floor muscles. Physiological modal data can monitor the activity of the pelvic floor muscles in real time using electromyographic sensors, while motion modal data can capture the patient's movement trajectory and posture using accelerometers. Video data can provide visual feedback on the patient's posture during training.

[0057] Extracting key information from the processed data is a key step. The key information here includes: heart rate, blood pressure, respiratory rate, muscle contraction or muscle relaxation data in physiological modal data, etc. These indicators can reflect the patient's physiological state. In motion modal data, such as gait cycle, step length, movement speed, acceleration, etc. in motion modal data, these features can reflect the patient's exercise ability and rehabilitation progress. In this embodiment, physiological modal data can extract the frequency and intensity of muscle contraction, and motion modal data can analyze the speed, acceleration and stability of movement. After multimodal data fusion analysis, the system can monitor the patient's rehabilitation status in real time. Based on the test results, the system can provide personalized feedback and suggestions to help patients adjust their training methods and intensity. The method of the present invention can more comprehensively evaluate the patient's rehabilitation status through multimodal data fusion and comprehensive utilization of physiological modal data and motion modal data, thereby improving the accuracy of detection.

[0058] Specifically, the pelvic floor rehabilitation training status detection method based on multimodal data fusion of an embodiment of the present invention is applied to the pelvic floor muscle repair scenario, and is used to fuse multiple modal data and calculate the user's training data under rehabilitation training in multiple dimensions, so as to detect the user's rehabilitation training in real time, and obtain comprehensive detection data based on multi-dimensional data fusion, so as to evaluate the user's rehabilitation training effect more comprehensively and comprehensively.

[0059] Before performing the pelvic floor rehabilitation training state detection method based on multimodal data fusion according to an embodiment of the present invention, it is necessary to collect the user's training data through various sensors, including installing multiple sensors on the patient, bioelectric signal sensors: such as surface electromyography sensors, which are used to monitor the electrical activity of the pelvic floor muscles in real time; and pressure sensors: placed in the pelvic floor area, which record the pressure changes under various postures in real time; and motion sensors: using accelerometers and gyroscopes to monitor the patient's movement trajectory and training movements to ensure that the dynamic performance during training is recorded. It also includes a pelvic floor muscle image state feature acquisition unit that can obtain images of the patient's pelvic floor muscles, and an intuitive display of the patient's pelvic floor structure can be obtained through the pelvic floor muscle images. By collecting data through multiple sensors, a comprehensive understanding of the patient's physiological and motor status can be achieved.

[0060] In this embodiment, the multimodal data collected includes motion modal data and physiological modal data. Among them, the motion modal data is collected by motion sensors, and the collected motion modal data specifically includes the following types of data:

[0061] Acceleration data: The three-axis acceleration values ​​(X, Y, and Z) can be used to monitor the patient's dynamic movements during training, including up and down, left and right, and front and back movements. Angular velocity data: The three-axis angular velocity values ​​(roll, pitch, and yaw) are used to measure the patient's rotational movements and posture changes during training. Displacement data: Based on the acceleration data, the patient's displacement can be calculated by integration to reflect the specific path of the training movement. Posture data: The body posture analyzed using sensor data, and information such as joint angles can be extracted through motion capture algorithms. Training movement frequency and duration: The execution frequency and duration of each training movement are recorded to facilitate the analysis of exercise intensity and endurance.

[0062] Physiological modality data is collected through bioelectrical signal sensors. The collected data includes electromyographic signals (EMGs), which are collected through surface electromyographic sensors and used to monitor the electrical activity of the pelvic floor muscles, including contraction and relaxation states, which can reflect muscle strength and endurance. Pressure distribution is collected through pressure sensors and used to record pressure changes in the pelvic floor area under different postures.

[0063] In addition, multimodal data can also include video data, which can be used as a tool for clinical assessment. By allowing the monitor to view video playback, the patient can receive more intuitive feedback. This helps patients understand their own athletic performance and make self-adjustments. Video data can provide additional information regarding subtle movements that sensors may not be able to accurately capture. For example, the tension and coordination of muscles during a specific movement may be difficult to capture using sensor data, but can be derived through video analysis. Multimodal data also includes pelvic floor imaging data, which intuitively provides a pelvic floor structure for the patient, providing a supplementary assessment of the test results after multimodal data fusion. That is, after obtaining the patient's pelvic floor muscle test results based on the fusion of motion modality data and physiological modality data, the patient's pelvic floor muscle test results are comprehensively evaluated in combination with the patient's pelvic floor imaging data, thereby forming a more targeted test result.

[0064] Specifically, video data can be used to extract key points of movement and posture changes through computer vision technology. This information provides the basis for subsequent state detection. Video data, along with motion modality data and physiological modality data, plays a synergistic role in pelvic floor rehabilitation training detection. Video data is used to obtain posture data of the user undergoing rehabilitation while in motion. By extracting key points of the user's motion posture, it is assessed whether the user's current motion posture meets the standard. If there is any abnormality in the motion posture, the user is prompted, thereby strengthening the supervision of the user's motion during the rehabilitation process.

[0065] Before using data for calculations, necessary data preprocessing is required. This is mainly because the collected multimodal data usually contain noise and omissions, so data preprocessing is needed. Through preprocessing, the accuracy of subsequent data analysis is improved and the interference of noise on the results is reduced.

[0066] Different preprocessing methods are used for different data. For physiological modality data, the preprocessing method is processed according to the following formula:

[0067] The physiological modality data is preprocessed using formula (1): Formula (1),

[0068] in, represents the preprocessed physiological modality data, represents the initial physiological modality data, Represents the initial physiological modality data The mean of Represents the initial physiological modality data The cutoff frequency, Represents the initial physiological modality data The average frequency of Represents the initial physiological modality data The standard deviation of .

[0069] In this embodiment, the physiological modality data includes: Electromyographic signals: collected by surface electromyographic sensors to monitor the electrical activity of the pelvic floor muscles, including contraction and relaxation states, which can reflect muscle strength and endurance. Pressure distribution: collected by pressure sensors to record pressure changes in the pelvic floor area under different postures.

[0070] Initial physiological modality data The purpose of preprocessing is to:

[0071] 1) Denoising: Because multimodal data acquisition can be affected by environmental interference, sensor errors, and other factors, the raw physiological modality data may contain a lot of noise. Preprocessing can reduce the impact of this noise on the results, making the data closer to the actual physiological state.

[0072] 2) Data smoothing: by applying the cutoff frequency and average frequency The ratio of can smooth the physiological signal, so as to better extract the real information of the signal.

[0073] 3) Standardization: By subtracting the mean and dividing by the standard deviation, the data are made to have a uniform scale, so that data under different conditions can be more effectively compared in subsequent analysis and modeling.

[0074] 4) Enhance signal characteristics: Through methods such as logarithmic transformation, signal characteristics can be enhanced, making the recognition of physiological activities higher during analysis.

[0075] The use of a method such as formula (1) to preprocess physiological modal data mainly considers the use of frequency-related weight adjustment without losing signal characteristics, thereby increasing the emphasis on the effective range of the signal while removing noise. For this preprocessing method, the ratio of the cutoff frequency to the average frequency of the data is used to affect the smoothness of the signal. This is more complex than the traditional simple mean smoothing or moving average, and can better adapt to the frequency characteristics of the data. In addition, the use of nonlinear processing, that is, the use of logarithmic functions and Kalman gains for enhancement and adjustment, may more effectively capture the nonlinear characteristics and dynamic changes of the signal under certain circumstances. The processing method of physiological modal data in this embodiment adopts a complex model and comprehensively uses multiple statistical features (such as mean, standard deviation, frequency factors, etc.) for data processing. Such a diversified processing method can improve the ability to analyze complex physiological signals.

[0076] It should be further explained that the physiological modality data in this embodiment is collected using corresponding sensors, such as bioelectric signal sensors: surface electromyography sensors (sEMG) are used to monitor the electrical activity of the pelvic floor muscles in real time, which can capture changes in electrical signals during muscle contraction and relaxation. Pressure sensors: Pressure sensors are placed in the pelvic floor area to monitor pressure changes under different postures. These sensors can provide data on the response of the pelvic floor muscles to different activities (such as walking, sitting, etc.). Motion sensors: Accelerometers and gyroscopes are installed to monitor the patient's movement trajectory and training movements. By recording dynamic performance through these sensors, the evaluation of training effects becomes more accurate.

[0077] Furthermore, the motion modal data is preprocessed using formula (2): Formula (2),

[0078] in represents the preprocessed motion modal data, represents the initial motion modal data, Represents the initial motion modal data The mean of Represents the initial motion modal data The cutoff frequency, Represents the initial motion modal data The average frequency of Represents the initial motion modal data The standard deviation of Represents the initial motion modal data The Kalman gain.

[0079] Motion modal data It is a multi-dimensional data set, usually represented as a matrix. Each dimension represents a different motion parameter or feature, specifically including the following types of data:

[0080] Acceleration data: The acceleration values ​​of the three axes (X, Y, Z) can be used to monitor the patient's dynamic movements during training, including up and down, left and right, and front and back movements.

[0081] Angular velocity data: Angular velocity values ​​along three axes (roll, pitch, and yaw) are used to measure the patient's rotational movements and posture changes during training.

[0082] Displacement data: Based on the acceleration data, the patient's displacement can be calculated by integration to reflect the specific path of the training movement.

[0083] Posture data: Body posture analyzed using sensor data, including information such as joint angles extracted through motion capture algorithms.

[0084] Training exercise frequency and duration: Record the frequency and duration of each training exercise to facilitate analysis of exercise intensity and endurance.

[0085] After preprocessing the motion and physiological modal data, the processor further extracts key information from the motion and physiological modal data to obtain at least one key modal feature from each of the physiological and motion modal data. This step is crucial for the multimodal data fusion method for detecting pelvic floor anti-toxic training status. To improve detection accuracy and reduce computational complexity, this embodiment extracts key modal feature data from the physiological and motion modal data for fusion. This accurately characterizes the pelvic floor status while improving computational efficiency and speed.

[0086] Specifically, key modal feature data of physiological modal data is extracted, including:

[0087] obtaining an electromyographic signal from the physiological modality data;

[0088] performing a preset process on the electromyographic signal to extract a characteristic frequency therefrom;

[0089] Performing a preset process on the amplitude value of the electromyography signal, and determining the amplitude value of the electromyography signal after the preset process as a muscle contraction strength index value;

[0090] The characteristic frequency and the muscle contraction strength index value are determined as key modal feature data of the physiological modal data.

[0091] In addition, key modal feature data of motion modal data are extracted, including:

[0092] The step of extracting key information of the multimodal data and obtaining key modal feature data of each modal data includes:

[0093] Extracting speed data and acceleration data of the user to be rehabilitated from the motion modality data;

[0094] Calculating a stability assessment value based on the rate of change of the acceleration data;

[0095] The velocity data, the acceleration data, and the stability evaluation value are determined as key modal feature data of the motion modal data.

[0096] In this embodiment, the frequency and intensity of muscle contraction can be obtained through electromyographic signals. In this embodiment, key modal feature data of each modal data needs to be extracted before fusing the multimodal data. These key modal feature data can be used to characterize the key factors of the pelvic floor muscle status.

[0097] Physiological modality data can be used to extract the frequency and intensity of muscle contractions, specifically through electromyography (EMG) signals. For example, frequency: The frequency of muscle contractions can be extracted from EMG data. By performing a fast Fourier transform (FFT) on the EMG signal, the primary frequency components in the signal can be analyzed to determine the muscle contraction frequency. Another example is muscle contraction strength: The strength of a muscle contraction can be measured by the amplitude of the EMG signal. A higher amplitude indicates greater muscle activity. The root mean square value (RMS) or mean absolute value (MAV) of the signal can be calculated as an indicator of muscle contraction strength.

[0098] Motion modal data can reflect speed, acceleration, and stability. When selecting key information, motion modal data is extracted from these key information: speed, acceleration, and stability. As for speed, the speed calculation can be obtained through acceleration sensor data. Acceleration data can be obtained directly from the accelerometer, and changes in movement can be monitored by recording the acceleration value (X, Y, Z axis) at each time point during the movement. If the acceleration increases suddenly, it may mean that the patient is making quick movements or suddenly changing posture. Stability is usually assessed by observing the rate of change of acceleration. In a stable state, the patient's acceleration changes should be smooth and small. If the acceleration value fluctuates violently at a certain moment, it may mean that the patient lacks movement coordination or has balance problems.

[0099] In actual rehabilitation training, situations where the monitor is mainly concerned with speed, acceleration, and stability data include:

[0100] Dynamic training: such as walking, running, going up and down stairs, etc., at this time the changes in speed and acceleration are particularly important and can reflect the patient's dynamic control ability.

[0101] Balance and coordination training: For example, standing on one leg, turning around, etc. In this case, stability data is extremely important and can help the monitor judge the patient's performance during balance training.

[0102] Load or resistance training: When increasing the training intensity or load, changes in acceleration and velocity can reflect the patient's adaptability and the training effect.

[0103] In addition to the key modal feature data described above, in other embodiments, the key modal feature data may also be selected from: pre-processed physiological modal data Select key data from the motion modality data, such as heart rate, blood pressure, respiratory rate, etc. These indicators can reflect the patient's physiological state. Extract key data from motion modality data: Motion feature selection: From the preprocessed motion modality data Key features such as gait cycle, step length, and movement speed are extracted from the data, which can reflect the patient's motor ability and rehabilitation progress.

[0104] After extracting the above-mentioned key modal feature data, the above-mentioned data needs to be converted into a data format so that different data can be effectively integrated. The specific data format conversion method is a common data format conversion method in the field. The ultimate goal is to effectively combine multiple data.

[0105] After the processor extracts key modal feature data from the multimodal data and performs necessary data format conversion so that different data can be combined and fused, the processor finally fuses the processed data using the fusion method of this embodiment. Specifically, in this embodiment, the key modal feature data of each modal data is fused to obtain fused multimodal data, including:

[0106] Processing the key modal feature data so that the key modal feature data of the physiological modal data and the key modal data of the motion modal data meet data fusion requirements;

[0107] The key modal data of the physiological modal data and the key modal data of the motion modal data are fused according to formula (3): Formula (3),

[0108] in, Represents the fused multimodal data, which combines the information of physiological modality data and motion modality data. represents the preprocessed physiological modality data, represents the preprocessed motion modal data, Represents preprocessed physiological modality data The weight adjustment factor, Represents the preprocessed motion modal data The weight adjustment factor, represents the Laplace distribution density function, j represents the preprocessed motion modal data The scale factor is used to control the data expansion and contraction in the data fusion process. a represents the preprocessed physiological modality data. The penalty factor, i represents the pre-processed motion modal data The penalty factor, t represents the initial physiological modality data and initial motion modal data The data collection time point, k represents the pre-processed physiological modality data The scaling factor of .

[0109] Data integration: Converting motion modality data into a format suitable for fusion, ensuring they can be effectively combined with physiological modality data. Data fusion: After extracting key data, the actual data fusion process is carried out.

[0110] In this embodiment, multimodal data fusion primarily considers data complementarity, specifically the complementarity between motion and physiological modal data. This is also the correlation between physiological and motion modal data: Physiological modal data, such as muscle electrical activity and heart rate, reflects the physiological state of muscles and their response to movement, while motion modal data, such as velocity, acceleration, and angle, describes actual movement behavior. Combining these two allows for a more comprehensive understanding of athletic performance and the underlying physiological mechanisms.

[0111] Furthermore, the fusion of multimodal data also considers the dynamic changes and stability of pelvic floor rehabilitation. Changes in movement state, such as fluctuations in velocity and acceleration, can directly affect the physiological parameters and functional state of the muscles. By fusing this data, the safety and effectiveness of exercise can be assessed.

[0112] In the embodiment of the present invention, the design basis for the fusion of multimodal data is mainly based on the following considerations: the introduction of weight adjustment factors α and β, α and β as weight adjustment factors, indicating the proportion of data in the fusion process. This design allows the importance of physiological modality data and motion modality data in the fusion to be flexibly adjusted according to specific application requirements. For example, in some cases, physiological modality data can be considered more important than motion modality data, and vice versa. Laplace distribution density function and scale factor j, using the Laplace distribution density function Smoothing the data can reduce the impact of noise on data fusion, making the final fusion data more stable and reliable. j acts as a scale factor. , which can control the scalability of motion modal data fusion. This enables automatic adaptation to changes and characteristics of different data sources during data fusion. In addition, penalty factors a and i are introduced to control the degree of influence of physiological modal data and motion modal data. These factors can be set to adjust the positive or negative impact of different data features on the final fusion result. For example, if a certain indicator deviates significantly from the normal range, the corresponding data weighting can be reduced to reduce the impact of outliers. Complex exponential transformation: Complex exponential transformation is applied to motion modal data to enhance the dynamic features in the motion modal data, so that the changes in motion state can be better reflected in the fused data.

[0113] The embodiment of the present invention adopts a multimodal data fusion design to ensure that the fusion result can effectively reflect the patient's recovery status by introducing weights, penalty factors and distribution characteristics, and provide support for subsequent status monitoring and analysis.

[0114] In another embodiment, after performing data fusion on the key modal feature data of each modal data to obtain fused multimodal data, the method further includes:

[0115] Formula (4) is used to perform data enhancement processing on the fused multimodal data: Formula (4),

[0116] in, represents the fused multimodal data, represents the enhanced fusion data, Represents the fused multimodal data frequency, Represents the fused multimodal data The phase offset, Represents the fused multimodal data The variance of Represents the fused multimodal data The mean of .

[0117] In this embodiment, data enhancement is used to introduce frequency The oscillation function of , makes data enhancement not limited to simple scale changes. This oscillation can perturb the data at different frequencies, increasing the diversity and complexity of the data. In addition, data enhancement includes phase bias , which makes the data generated have a certain phase rotation characteristic. This enhances the model's adaptability to data changes and helps improve its performance when dealing with rapidly changing or periodic features. It provides information from different sources, and by enhancing this information, it can further improve the model's ability to capture various input characteristics.

[0118] Finally, the processor inputs the fused multimodal data into the rehabilitation detection calculation model to obtain the rehabilitation detection results of the user to be rehabilitated. The specific processing method includes:

[0119] Formula (5) is used to calculate the multimodal data to obtain the rehabilitation test results of the user to be rehabilitated: Formula (5),

[0120] Among them, C represents the rehabilitation status detection index, which represents the patient's rehabilitation status score. represents the enhanced fused multimodal data, represents the preprocessed physiological modality data, represents the preprocessed motion modal data, Represents the fused multimodal data The impact factor, Represents preprocessed physiological modality data and preprocessed motion modal data The attenuation factor, Indicates the recovery status threshold.

[0121] if Indicates good condition, continue to maintain training intensity, if It is recommended to reduce the intensity of training and take rest. Based on the test results, personalized feedback and suggestions can be provided to help patients adjust their training methods and intensity.

[0122] In another embodiment of the present invention, referring to Figure 2 The present application also provides a flowchart of a training method for a rehabilitation detection calculation model in a pelvic floor rehabilitation training state detection method based on multimodal data fusion, wherein the training method includes:

[0123] Step 110: Acquire multimodal fusion data of the user's historical rehabilitation training process;

[0124] Step 120, inputting the multimodal fusion data into the rehabilitation detection calculation model;

[0125] Step 130, obtaining the detection result output by the model;

[0126] Step 140, calculating the deviation between the output detection result and the actual result;

[0127] Step 150, adjusting the parameters of the rehabilitation detection calculation model according to the deviation value;

[0128] Steps 110 to 150 are repeated until the deviation between the detection result obtained by the rehabilitation detection calculation model and the actual result is within a preset range.

[0129] The method of this embodiment is intended to use an artificial intelligence algorithm to calculate the pelvic floor detection results through model calculation, thereby improving the intelligence and efficiency of the calculation.

[0130] Figure 3 This is a structural block diagram of a pelvic floor rehabilitation training status detection system based on multimodal data fusion provided in an embodiment of the present application. Figure 3 As shown, an embodiment of the present application provides a pelvic floor rehabilitation training status detection system based on multimodal data fusion, which may include:

[0131] Memory 310 configured to store instructions; and

[0132] The processor 320 is configured to call instructions from the memory 310 and implement the above-mentioned pelvic floor rehabilitation training status detection method based on multimodal data fusion when executing the instructions.

[0133] An embodiment of the present application further provides a machine-readable storage medium having stored thereon instructions for enabling a machine to execute the above-mentioned pelvic floor rehabilitation training status detection method based on multimodal data fusion.

[0134] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0135] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0136] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0137] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0138] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0139] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0140] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can be implemented using any method or technology for information storage. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change RAM (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0141] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0142] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A pelvic floor rehabilitation training status detection method based on multimodal data fusion, characterized in that: include: Acquiring multimodal data of the user to be rehabilitated collected during rehabilitation training, wherein the multimodal data includes motion modal data and physiological modal data; Extracting key information of the multimodal data to obtain key modal feature data of each modal data in the multimodal data; Performing data fusion on the key modal feature data of each modal data to obtain fused multimodal data; The fused multimodal data is input into the rehabilitation detection calculation model to obtain the rehabilitation detection results of the user to be rehabilitated. Wherein, before extracting the key information of the multimodal data, the pelvic floor rehabilitation training state detection method based on multimodal data fusion further includes: The physiological modality data is preprocessed using formula (1): Formula (1) in, represents the preprocessed physiological modality data, represents the initial physiological modality data, Represents the initial physiological modality data The mean of Represents the initial physiological modality data The cut-off frequency, Represents the initial physiological modality data The average frequency of Represents the initial physiological modality data The standard deviation of .

2. The pelvic floor rehabilitation training state detection method based on multimodal data fusion according to claim 1, characterized in that: Extracting key information of the multimodal data to obtain key modal feature data of each modal data in the multimodal data includes: acquiring an electromyographic signal from the physiological modality data; performing a preset process on the electromyographic signal to extract a characteristic frequency therefrom; Performing preset processing on the amplitude value of the electromyography signal, and determining the amplitude value of the electromyography signal after the preset processing as the muscle contraction intensity index value; The characteristic frequency and the muscle contraction strength index value are determined as key modal feature data of the physiological modal data.

3. The pelvic floor rehabilitation training state detection method based on multimodal data fusion according to claim 2, characterized in that: Extracting key information of the multimodal data to obtain key modal feature data of each modal data in the multimodal data also includes: Extracting speed data and acceleration data of the user to be rehabilitated from the motion modality data; Calculating a stability evaluation value according to the rate of change of the acceleration data; The velocity data, the acceleration data, and the stability evaluation value are determined as key modal feature data of the motion modal data.

4. The pelvic floor rehabilitation training state detection method based on multimodal data fusion according to claim 1, characterized in that: Before extracting the key information of the multimodal data, the pelvic floor rehabilitation training state detection method based on multimodal data fusion further includes: The motion modal data is preprocessed using formula (2): Formula (2) in, represents the preprocessed motion modal data, represents the initial motion modal data, Represents the initial motion modal data The mean of Represents the initial motion modal data The cut-off frequency, Represents the initial motion modal data The average frequency of Represents the initial motion modal data The standard deviation of Represents the initial motion modal data The Kalman gain.

5. The pelvic floor rehabilitation training state detection method based on multimodal data fusion according to claim 4, characterized in that: The key modal feature data of each modal data is subjected to data fusion to obtain fused multimodal data, including: Processing the key modal feature data so that the key modal feature data of the physiological modal data and the key modal data of the motion modal data meet data fusion requirements; The key modality data of the physiological modality data and the key modality data of the motion modality data are fused according to formula (3): Formula (3) in, Represents the fused multimodal data, which combines the information of physiological modality data and motion modality data. represents the preprocessed physiological modality data, represents the preprocessed motion modal data, Represents the preprocessed physiological modality data The weight adjustment factor is Represents the preprocessed motion modal data The weight adjustment factor is represents the Laplace distribution density function, j represents the preprocessed motion modal data The scale factor is used to control the data expansion and contraction in the data fusion process. a represents the preprocessed physiological modality data. The penalty factor is i, i represents the preprocessed motion modal data The penalty factor is t, which represents the initial physiological modality data. and initial motion modal data The data collection time point, k represents the preprocessed physiological modality data The scaling factor of .

6. The pelvic floor rehabilitation training state detection method based on multimodal data fusion according to claim 5, characterized in that: After the key modal feature data of each modal data is fused to obtain fused multimodal data, the pelvic floor rehabilitation training state detection method based on multimodal data fusion further includes: Formula (4) is used to perform data enhancement processing on the fused multimodal data: Formula (4) in, represents the fused multimodal data, represents the enhanced fusion data, Represents the fused multimodal data The frequency, Represents the fused multimodal data The phase offset, Represents the fused multimodal data The variance of Represents the fused multimodal data The mean of .

7. The pelvic floor rehabilitation training state detection method based on multimodal data fusion according to claim 6, characterized in that: The fused multimodal data is input into the rehabilitation detection calculation model to obtain the rehabilitation detection results of the user to be rehabilitated, including: The rehabilitation detection calculation model uses formula (5) to calculate the multimodal data to obtain the rehabilitation detection result of the user to be rehabilitated: Formula (5) Among them, C represents the rehabilitation status detection index, which represents the patient's rehabilitation status score. represents the enhanced fused multimodal data, represents the preprocessed physiological modality data, represents the preprocessed motion modal data, Represents the fused multimodal data The impact factor, Represents the preprocessed physiological modality data and preprocessed motion modal data The attenuation factor, Represents the recovery status threshold.

8. The pelvic floor rehabilitation training status detection system based on multimodal data fusion is characterized by: include: A memory storing instructions; as well as A processor, wherein the processor executes the instruction to implement the pelvic floor rehabilitation training state detection method based on multimodal data fusion according to any one of claims 1-7.

Images