Pelvic floor rehabilitation training status detection method and system based on multimodal data fusion

Abstract

The present application relates to the technical field of pelvic floor rehabilitation detection, and discloses a pelvic floor rehabilitation training state detection method and system based on multimodal data fusion. The method comprises: obtaining multimodal data of a user to be rehabilitated collected under a rehabilitation training state, wherein the multimodal data comprises motion modal data and physiological modal data; extracting key information of the multimodal data to obtain key modal feature data of each modal data in the multimodal data; fusing the key modal feature data of each modal data to obtain fused multimodal data; inputting the fused multimodal data into a rehabilitation detection calculation model to obtain rehabilitation detection results of the user to be rehabilitated. The method of the present invention can more comprehensively evaluate the patient's rehabilitation state by fusion of multimodal data and comprehensive utilization of physiological modal data and motion modal data, thereby improving the accuracy of detection.

Description

Technical Field

[0001] The present application relates to the technical field of pelvic floor rehabilitation detection technology, and specifically to a pelvic floor rehabilitation training status detection method and system based on multimodal data fusion. Background Art

[0002] Pelvic floor dysfunction refers to various clinical symptoms caused by dysfunction of the pelvic floor muscles, such as urinary incontinence, dysuria, constipation, pelvic organ prolapse, etc. These problems not only affect the patient's physical health, but also have a serious impact on their psychological and social life. Therefore, the research and application of pelvic floor rehabilitation treatment is particularly important.

[0003] At present, pelvic floor rehabilitation treatment mainly helps patients restore the function of pelvic floor muscles through physical therapy, behavioral therapy and biofeedback. Studies have shown that early intervention and systematic rehabilitation training can significantly improve patients' symptoms and improve their quality of life. However, traditional pelvic floor rehabilitation training often relies on patients' self-perception and subjective feedback, lacks scientific monitoring methods and real-time data monitoring and feedback mechanisms, which leads to large individual differences in training effects, making it difficult to evaluate training effects and achieve precise rehabilitation goals.

[0004] In view of this, there is an urgent need for a pelvic floor rehabilitation training status detection method based on multimodal data fusion that can solve the above problems. Summary of the invention

[0005] In order to solve the above technical problems, the purpose of the embodiments of the present application is to provide a pelvic floor rehabilitation training state detection method based on multimodal data fusion, a processor and a machine-readable storage medium.

[0006] In order to achieve the above-mentioned purpose, the first aspect of the present application provides a pelvic floor rehabilitation training state detection method based on multimodal data fusion, comprising:

[0007] Acquiring multimodal data of the user to be rehabilitated collected during rehabilitation training, wherein the multimodal data includes motion modal data and physiological modal data;

[0008] Extracting key information of the multimodal data to obtain key modal feature data of each modal data in the multimodal data;

[0009] The key modal feature data of each modal data is fused to obtain fused multimodal data;

[0010] The fused multimodal data is input into the rehabilitation detection calculation model to obtain the rehabilitation detection results of the user to be rehabilitated.

[0011] Preferably, extracting key information of the multimodal data to obtain key modal feature data of each modal data in the multimodal data includes:

[0012] acquiring an electromyographic signal from the physiological modality data;

[0013] performing a preset process on the electromyographic signal to extract a characteristic frequency therefrom;

[0014] Performing preset processing on the amplitude value of the electromyography signal, and determining the amplitude value of the electromyography signal after the preset processing as the muscle contraction intensity index value;

[0015] The characteristic frequency and the muscle contraction strength index value are determined as key modal feature data of the physiological modal data.

[0016] Preferably, extracting key information of the multimodal data to obtain key modal feature data of each modal data in the multimodal data includes:

[0017] Extracting speed data and acceleration data of the user to be rehabilitated from the motion modality data;

[0018] Calculating a stability evaluation value according to the rate of change of the acceleration data;

[0019] The velocity data, the acceleration data, and the stability evaluation value are determined as key modal feature data of the motion modal data.

[0020] Preferably, before extracting the key information of the multimodal data, the method further comprises:

[0021] The physiological modality data is preprocessed using formula (1):

[0022] Formula (1),

[0023] in, represents the preprocessed physiological modality data, represents the initial physiological modality data, Represents the initial physiological modality data The mean of Represents the initial physiological modality data The cut-off frequency, Represents the initial physiological modality data The average frequency of Represents the initial physiological modality data The standard deviation of .

[0024] Preferably, before extracting the key information of the multimodal data, the method further comprises:

[0025] The motion modal data is preprocessed using formula (2):

[0026] Formula (2),

[0027] in, represents the preprocessed motion modal data, represents the initial motion modal data, Represents the initial motion modal data The mean of Represents the initial motion modal data The cut-off frequency, Represents the initial motion modal data The average frequency of Represents the initial motion modal data The standard deviation of Represents the initial motion modal data The Kalman gain.

[0028] Preferably, the key modal feature data of each modal data is fused to obtain fused multimodal data, including:

[0029] Processing the key modal feature data so that the key modal feature data of the physiological modal data and the key modal data of the motion modal data meet data fusion requirements;

[0030] The key modality data of the physiological modality data and the key modality data of the motion modality data are fused according to formula (3):

[0031] Formula (3),

[0032] in, Represents the fused multimodal data, which combines the information of physiological modality data and motion modality data. represents the preprocessed physiological modality data, represents the preprocessed motion modal data, Represents the preprocessed physiological modality data The weight adjustment factor is Represents the preprocessed motion modal data The weight adjustment factor is represents the Laplace distribution density function, j represents the preprocessed motion modal data The scale factor is used to control the data expansion and contraction in the data fusion process. a represents the preprocessed physiological modality data. The penalty factor is i, i represents the preprocessed motion modal data The penalty factor is t, which represents the initial physiological modality data. and initial motion modal data The data collection time point, k represents the preprocessed physiological modality data The scaling factor of .

[0033] Preferably, after fusing the key modal feature data of each modal data to obtain fused multimodal data, the method further comprises:

[0034] Formula (4) is used to perform data enhancement processing on the fused multimodal data:

[0035] Formula (4),

[0036] in, represents the fused multimodal data, represents the enhanced fusion data, Represents the fused multimodal data The frequency, Represents the fused multimodal data The phase offset, Represents the fused multimodal data The variance of Represents the fused multimodal data The mean of .

[0037] Preferably, the fused multimodal data is input into a rehabilitation detection calculation model to obtain rehabilitation detection results of the user to be rehabilitated, including:

[0038] The rehabilitation detection calculation model uses formula (5) to calculate the multimodal data to obtain the rehabilitation detection result of the user to be rehabilitated:

[0039] Formula (5),

[0040] Among them, C represents the rehabilitation status detection index, which represents the patient's rehabilitation status score. represents the enhanced fused multimodal data, represents the preprocessed physiological modality data, represents the preprocessed motion modal data, Represents the fused multimodal data The impact factor, Represents the preprocessed physiological modality data and preprocessed motion modal data The attenuation factor, Represents the recovery status threshold.

[0041] The second aspect of the present application provides a pelvic floor rehabilitation training status detection system based on multimodal data fusion, comprising:

[0042] a memory configured to store instructions; and

[0043] The processor is configured to call instructions from the memory and implement the above-mentioned pelvic floor rehabilitation training state detection method based on multimodal data fusion when executing the instructions.

[0044] A third aspect of the present application provides a machine-readable storage medium having instructions stored thereon, the instructions being used to enable a machine to execute the above-mentioned pelvic floor rehabilitation training state detection method based on multimodal data fusion.

[0045] Through the above technical scheme, the multimodal data of the user to be rehabilitated collected under the rehabilitation training state is obtained, and the multimodal data includes motion modal data and physiological modal data; the key information of the multimodal data is extracted to obtain the key modal feature data of each modal data in the multimodal data; the key modal feature data of each modal data is fused to obtain the fused multimodal data; the fused multimodal data is input into the rehabilitation detection calculation model to obtain the rehabilitation detection results of the user to be rehabilitated. The method of the present invention can more comprehensively evaluate the patient's rehabilitation status through multimodal data fusion and comprehensive use of physiological modal data and motion modal data, thereby improving the accuracy of detection; the method of the present invention can monitor the state changes during the rehabilitation training process, and provide timely feedback to the patient to help him adjust the training plan and ensure the effectiveness of the rehabilitation training.

[0046] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present application, but do not constitute a limitation on the embodiments of the present application. In the accompanying drawings:

[0048] Figure 1 A flowchart of a pelvic floor rehabilitation training state detection method based on multimodal data fusion provided in an embodiment of the present application;

[0049] Figure 2 A flowchart of a training method for a rehabilitation detection calculation model in a pelvic floor rehabilitation training state detection method based on multimodal data fusion provided in an embodiment of the present application;

[0050] Figure 3A structural block diagram of a pelvic floor rehabilitation training status detection system based on multimodal data fusion provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the specific implementation methods described herein are only used to illustrate and explain the embodiments of the present application, and are not used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0052] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back...), such directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0053] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist, and is not in the flow chart of the pelvic floor rehabilitation training state detection method based on multimodal data fusion required by the present application.

[0054] like Figure 1 As shown, the pelvic floor rehabilitation training state detection method based on multimodal data fusion in an embodiment of the present invention may include the following steps:

[0055] Step 10, acquiring multimodal data of the user to be rehabilitated collected during rehabilitation training, wherein the multimodal data includes motion modal data and physiological modal data;

[0056] Step 20, extracting key information of the multimodal data to obtain key modal feature data of each modal data in the multimodal data;

[0057] Step 30, fusing the key modal feature data of each modal data to obtain fused multimodal data;

[0058] Step 40: input the fused multimodal data into a rehabilitation test calculation model to obtain a rehabilitation test result for the user to be rehabilitated.

[0059] Traditional diagnostic methods for pelvic floor dysfunction often focus on only a single indicator, which makes it difficult to comprehensively assess the functional status of the pelvic floor muscles and has significant limitations.

[0060] The rise of multimodal data fusion With the development of sensor technology, data processing technology and artificial intelligence, multimodal data fusion technology has gradually been introduced into the field of medical health. Multimodal data fusion refers to the integration of data from different sources to obtain more comprehensive and accurate information. In the field of pelvic floor rehabilitation, the use of a variety of sensors, such as bioelectric signal sensors, pressure sensors, motion sensors, etc. for data collection can monitor the patient's training status in real time, thereby providing a scientific basis for rehabilitation training. The application of intelligent algorithms based on data fusion provides new possibilities for the state detection of pelvic floor rehabilitation training. Through technologies such as machine learning and deep learning, the collected multimodal data can be analyzed to identify the user's training mode, intensity and effect. This intelligent monitoring can not only improve the scientificity and accuracy of training, but also provide patients with personalized rehabilitation advice.

[0061] In an embodiment of the present invention, in pelvic floor rehabilitation training, multimodal data needs to be collected through wearable devices and cameras first. Pelvic floor electromyographic signals are of great significance for judging the functional state of the patient's pelvic floor muscles. Physiological modal data can monitor the activity of the pelvic floor muscles in real time through electromyographic sensors, while motion modal data can capture the patient's movement trajectory and posture through accelerometers. Video data can provide visual feedback of the patient's posture during training.

[0062] Extracting key information from the processed data is a key step. The key information here includes: heart rate, blood pressure, respiratory rate, muscle contraction or muscle relaxation data in physiological modal data, etc. These indicators can reflect the patient's physiological state. In motion modal data, such as gait cycle, step length, movement speed, acceleration, etc. in motion modal data, these features can reflect the patient's motor ability and rehabilitation progress. In this embodiment, physiological modal data can extract the frequency and intensity of muscle contraction, and motion modal data can analyze the speed, acceleration and stability of movement. After multimodal data fusion analysis, the system can monitor the patient's rehabilitation status in real time. Based on the test results, the system can provide personalized feedback and suggestions to help patients adjust training methods and intensity. The method of the present invention can more comprehensively evaluate the patient's rehabilitation status by multimodal data fusion and comprehensive utilization of physiological modal data and motion modal data, thereby improving the accuracy of detection.

[0063] Specifically, the pelvic floor rehabilitation training status detection method based on multimodal data fusion of an embodiment of the present invention is applied to the pelvic floor muscle repair scenario, and is used to fuse multiple modal data and calculate the user's training data under rehabilitation training in multiple dimensions, so as to detect the user's rehabilitation training in real time, and obtain comprehensive detection data based on multi-dimensional data fusion, so as to evaluate the user's rehabilitation training effect more comprehensively and comprehensively.

[0064] Before performing the pelvic floor rehabilitation training state detection method based on multimodal data fusion of the embodiment of the present invention, it is necessary to collect the user's training data through various sensors, including installing multiple sensors and bioelectric signal sensors on the patient: such as surface electromyography sensors, which are used to monitor the electrical activity of the pelvic floor muscles in real time; and pressure sensors: placed in the pelvic floor area, recording the pressure changes in various postures in real time; and motion sensors: using accelerometers and gyroscopes to monitor the patient's movement trajectory and training movements to ensure that the dynamic performance during training is recorded. It also includes a pelvic floor muscle image state feature acquisition unit that can obtain the patient's pelvic floor muscle image, and the pelvic floor muscle image can obtain an intuitive display of the patient's pelvic floor structure. By collecting data through multiple sensors, the patient's physiological and motor status can be fully understood.

[0065] In this embodiment, the collected multimodal data includes motion modal data and physiological modal data. Among them, the motion modal data is collected by a motion sensor, and the collected motion modal data specifically includes the following types of data:

[0066] Acceleration data: The acceleration values ​​of the three axes (X, Y, and Z) can be used to monitor the patient's dynamic movements during training, including up and down, left and right, and front and back movements. Angular velocity data: The angular velocity values ​​of the three axes (roll, pitch, and yaw) are used to measure the patient's rotational movements and posture changes during training. Displacement data: Based on the acceleration data, the patient's displacement can be calculated by integration to reflect the specific path of the training movement. Posture data: Body posture analyzed using sensor data, joint angles and other information extracted by motion capture algorithms. Training movement frequency and duration: Record the execution frequency and duration of each training movement to facilitate the analysis of exercise intensity and endurance.

[0067] Physiological modal data is collected through bioelectric signal sensors. The collected data includes electromyographic signals: collected through surface electromyographic sensors to monitor the electrical activity of the pelvic floor muscles, including contraction and relaxation states, which can reflect the strength and endurance of the muscles. Pressure distribution: collected through pressure sensors to record the pressure changes in the pelvic floor area under different postures.

[0068] In addition, multimodal data can also include video data, which can be used as a tool for clinical evaluation. By monitoring the video playback, patients can get more intuitive feedback. This helps patients understand their own athletic performance and make self-adjustments. Video data can provide additional information for some subtle movements that sensors may not be able to accurately capture. For example, the tension and coordination of muscles in a particular movement may be difficult to capture through sensor data, but can be obtained through video analysis. Multimodal data also includes pelvic floor imaging data, which intuitively gives the patient's pelvic floor structure through pelvic floor imaging, and supplements the evaluation of the test results after multimodal data fusion. That is, after the patient's pelvic floor muscle test results are obtained based on the fusion of motion modality data and physiological modality data, the patient's pelvic floor muscle test results are comprehensively evaluated in combination with the patient's pelvic floor imaging data, so as to form a more targeted test result.

[0069] Specifically, video data can be used to extract key points and posture changes of movement through computer vision technology. This information provides a basis for subsequent state detection. Video data, motion modality data, and physiological modality data work together to play a role in pelvic floor rehabilitation training detection. The posture data of the user to be rehabilitated in motion is obtained through video data. By extracting the key points of the user's motion posture, it is evaluated whether the user's current motion posture meets the standard. If the motion posture is abnormal, a prompt will be issued to the user, which can strengthen the user's motion supervision during the rehabilitation process.

[0070] Before using the data for calculation, necessary data preprocessing is required. This is mainly because the collected multimodal data usually contain noise and omissions, so data preprocessing is needed. Through preprocessing, the accuracy of subsequent data analysis is improved and the interference of noise on the results is reduced.

[0071] Different preprocessing methods are used for different data. For physiological modality data, the preprocessing method is processed according to the following formula:

[0072] The physiological modality data is preprocessed using formula (1):

[0073] Formula (1),

[0074] in, represents the preprocessed physiological modality data, represents the initial physiological modality data, Represents the initial physiological modality data The mean of Represents the initial physiological modality data The cut-off frequency, Represents the initial physiological modality data The average frequency of Represents the initial physiological modality data The standard deviation of .

[0075] In this embodiment, the physiological modality data includes: electromyographic signals: collected by surface electromyographic sensors, used to monitor the electrical activity of the pelvic floor muscles, including contraction and relaxation states, which can reflect the strength and endurance of the muscles. Pressure distribution: collected by pressure sensors, used to record the pressure changes in the pelvic floor area under different postures.

[0076] Initial physiological modality data The purpose of preprocessing is to:

[0077] 1) De-noising: Since the multimodal data collection process may be affected by environmental interference, sensor errors and other factors, there may be a lot of noise in the original physiological modality data. Through preprocessing, the impact of these noises on the results can be reduced, making the data closer to the real physiological state.

[0078] 2) Data smoothing: by applying a cutoff frequency and the average frequency The ratio of can be used to smooth the physiological signal, so as to better extract the real information of the signal.

[0079] 3) Standardization: By subtracting the mean and dividing by the standard deviation, the data is made to have a uniform scale, so that data under different conditions can be more effectively compared in subsequent analysis and modeling.

[0080] 4) Enhance signal characteristics: Through methods such as logarithmic transformation, signal characteristics can be enhanced to make the recognition of physiological activities higher during analysis.

[0081] The method of using formula (1) to preprocess physiological modal data mainly considers using frequency-related weight adjustment without losing signal characteristics, and increasing the emphasis on the effective range of the signal while denoising. For this preprocessing method, the ratio of the cutoff frequency to the average frequency of the data is used to affect the smoothness of the signal, which is more complex than the traditional simple mean smoothing or moving average, and can better adapt to the frequency characteristics of the data. In addition, the use of nonlinear processing, that is, the use of logarithmic functions and Kalman gains for enhancement and adjustment, may more effectively capture the nonlinear characteristics and dynamic changes of the signal under certain circumstances. The processing method of physiological modal data in this embodiment adopts a complex model and comprehensively uses a variety of statistical features (such as mean, standard deviation, frequency factors, etc.) for data processing. Such a diversified processing method can improve the ability to analyze complex physiological signals.

[0082] It should be further explained that the collection of physiological modal data in this embodiment is carried out using corresponding sensors, such as bioelectric signal sensors: surface electromyography sensors (sEMG) are used to monitor the electrical activity of the pelvic floor muscles in real time, and the changes in electrical signals during muscle contraction and relaxation can be captured. Pressure sensor: a pressure sensor is placed in the pelvic floor area to monitor pressure changes in different postures. These sensors can provide data on the response of the pelvic floor muscles to different activities (such as walking, sitting, etc.). Motion sensor: an accelerometer and a gyroscope are installed to monitor the patient's movement trajectory and training movements. By recording dynamic performance through these sensors, the evaluation of training effects becomes more accurate.

[0083] Furthermore, the motion modal data is preprocessed using formula (2):

[0084] Formula (2),

[0085] in represents the preprocessed motion modal data, represents the initial motion modal data, Represents the initial motion modal data The mean of Represents the initial motion modal data The cut-off frequency, Represents the initial motion modal data The average frequency of Represents the initial motion modal data The standard deviation of Represents the initial motion modal data The Kalman gain.

[0086] Motion modal data It is a multi-dimensional data set, which can usually be represented as a matrix. Each dimension represents a different motion parameter or feature, including the following types of data:

[0087] Acceleration data: The acceleration values ​​of the three axes (X, Y, Z) can be used to monitor the patient's dynamic movements during training, including up and down, left and right, and front and back movements.

[0088] Angular velocity data: Angular velocity values ​​in three axes (roll, pitch, yaw) are used to measure the patient's rotational movements and posture changes during training.

[0089] Displacement data: Based on the acceleration data, the patient's displacement can be calculated by integration, reflecting the specific path of the training movement.

[0090] Posture data: Body posture analyzed using sensor data, information such as joint angles extracted through motion capture algorithms.

[0091] Exercise frequency and duration: Record the frequency and duration of each exercise to analyze exercise intensity and endurance.

[0092] After preprocessing the motion modal data and the physiological modal data, the processor further extracts key information of the motion modal data and the physiological modal data to obtain at least one key modal feature data of the physiological modal data and the motion modal data. This step is a key step for the entire multimodal data fusion pelvic floor anti-toxic training state detection method. In order to reduce the amount of calculation on the basis of improving the accuracy of detection, in this embodiment, key modal feature data is extracted from the physiological modal data and the motion modal data for fusion, which can accurately characterize and reflect the pelvic floor state while improving the calculation efficiency and speed.

[0093] Specifically, key modal feature data of physiological modal data are extracted, including:

[0094] acquiring an electromyographic signal from the physiological modality data;

[0095] performing a preset process on the electromyographic signal to extract a characteristic frequency therefrom;

[0096] Performing preset processing on the amplitude value of the electromyography signal, and determining the amplitude value of the electromyography signal after the preset processing as the muscle contraction intensity index value;

[0097] The characteristic frequency and the muscle contraction strength index value are determined as key modal feature data of the physiological modal data.

[0098] In addition, the key modal feature data of the motion modal data are extracted, including:

[0099] The step of extracting key information of the multimodal data and obtaining key modal feature data of each modal data comprises:

[0100] Extracting speed data and acceleration data of the user to be rehabilitated from the motion modality data;

[0101] Calculating a stability evaluation value according to the rate of change of the acceleration data;

[0102] The velocity data, the acceleration data, and the stability evaluation value are determined as key modal feature data of the motion modal data.

[0103] In this embodiment, the frequency and intensity of muscle contraction can be obtained through electromyographic signals. In this embodiment, key modal feature data of each modal data needs to be extracted before fusing the multimodal data. These key modal feature data can be used to characterize the key factors of the pelvic floor muscle state.

[0104] Physiological modal data can be used to extract the frequency and intensity of muscle contractions, which can be specifically extracted from electromyography (EMG) signals. For example, frequency: The frequency of muscle contractions can be extracted from EMG data. By performing a fast Fourier transform (FFT) on the EMG signal, the main frequency components in the signal can be analyzed to determine the contraction frequency of the muscle. Another example is the strength of muscle contraction: The strength of muscle contraction can be measured by the amplitude of the EMG signal. A higher amplitude indicates a greater intensity of muscle activity. The root mean square value (RMS) or mean absolute value (MAV) of the signal can be calculated as an indicator of muscle contraction strength.

[0105] Motion modal data can reflect speed, acceleration and stability. When selecting key information, the motion modal data is extracted from the key information of speed, acceleration and stability. For speed, the calculation of speed can be obtained through the acceleration sensor data. Acceleration data can be obtained directly from the accelerometer to monitor changes in movement by recording the acceleration value (X, Y, Z axis) at each time point in the movement. If the acceleration increases suddenly, it may mean that the patient is making fast movements or suddenly changing postures. Stability is usually assessed by observing the rate of change of acceleration. In a stable state, the patient's acceleration changes should be smooth and small. If the acceleration value fluctuates violently at a certain moment, it may mean that the patient lacks movement coordination or has balance problems.

[0106] In actual rehabilitation training, situations where the monitor is mainly concerned with speed, acceleration, and stability data include:

[0107] Dynamic training: such as walking, running, going up and down stairs, etc., at this time, the changes in speed and acceleration are particularly important and can reflect the patient's dynamic control ability.

[0108] Balance and coordination training: For example, standing on one leg, turning around, etc. In this case, stability data is extremely important and can help the monitor judge the patient's performance during balance training.

[0109] Load or resistance training: When increasing the training intensity or load, changes in acceleration and velocity can reflect the patient's adaptability and the training effect.

[0110] In addition to the key modal feature data described above, in other embodiments, the key modal feature data may also be selected from: pre-processed physiological modal data Select key data from the preprocessed motion modality data, such as heart rate, blood pressure, respiratory rate, etc. These indicators can reflect the patient's physiological state. Extract key data from motion modality data: Motion feature selection: Extract key data from the preprocessed motion modality data Key features such as gait cycle, step length, movement speed, etc. are extracted from the data, which can reflect the patient's motor ability and rehabilitation progress.

[0111] After extracting the above key modal feature data, the above data needs to be converted into a data format so that different data can be effectively integrated. The specific data format conversion method is a common data format conversion method in the field. The ultimate goal is to effectively combine multiple data.

[0112] After the processor extracts key modal feature data from the multimodal data and performs necessary data format conversion so that different data can be combined and fused with each other, the processor finally fuses the processed data using the fusion method of this embodiment. Specifically, in this embodiment, the key modal feature data of each modal data is fused to obtain fused multimodal data, including:

[0113] Processing the key modal feature data so that the key modal feature data of the physiological modal data and the key modal data of the motion modal data meet data fusion requirements;

[0114] The key modality data of the physiological modality data and the key modality data of the motion modality data are fused according to formula (3):

[0115] Formula (3),

[0116] in, Represents the fused multimodal data, which combines the information of physiological modality data and motion modality data. represents the preprocessed physiological modality data, represents the preprocessed motion modal data, Represents the preprocessed physiological modality data The weight adjustment factor is Represents the preprocessed motion modal data The weight adjustment factor is represents the Laplace distribution density function, j represents the preprocessed motion modal data The scale factor is used to control the data expansion and contraction in the data fusion process. a represents the preprocessed physiological modality data. The penalty factor is i, i represents the preprocessed motion modal data The penalty factor is t, which represents the initial physiological modality data. and initial motion modal data The data collection time point, k represents the preprocessed physiological modality data The scaling factor of .

[0117] Data integration: Convert the motion modality data into a format suitable for fusion, ensuring that they can be effectively combined with the physiological modality data. Data fusion: After extracting the key data, the actual data fusion process is carried out.

[0118] In this embodiment, the fusion of multimodal data mainly considers the complementarity of data, that is, the complementarity of motion modal data and physiological modal data, that is, the association between physiological modal data and motion modal data: physiological modal data, such as muscle electrical activity, heart rate, etc., can reflect the physiological state of muscles and their response to movement, while motion modal data, such as speed, acceleration, angle, etc., describe actual movement behavior. The combination of the two can provide a more comprehensive understanding of sports performance and the physiological mechanism behind it.

[0119] In addition, the fusion of multimodal data also takes into account the dynamic changes and stability during pelvic floor rehabilitation: changes in movement state, such as fluctuations in speed and acceleration, can directly affect the physiological parameters and functional state of the muscles. By fusing these data, the safety and effectiveness of the exercise can be evaluated.

[0120] In the embodiment of the present invention, the design basis for fusing multimodal data is mainly based on the following considerations: introducing weight adjustment factors α and β, α and β as weight adjustment factors, indicating the proportion of data in the fusion process. This design allows the importance of physiological modality data and motion modality data in fusion to be flexibly adjusted according to specific application requirements. For example, in some cases, physiological modality data can be considered more important than motion modality data, and vice versa. Laplace distribution density function and scale factor j, using the Laplace distribution density function Smoothing the data can reduce the impact of noise on data fusion, making the final fused data more stable and reliable. j acts as a scale factor. , the scalability of motion modal data fusion can be controlled. This enables automatic adaptation to changes and characteristics of different data sources during data fusion. In addition, penalty factors a and i are introduced to control the degree of influence of physiological modal data and motion modal data. These factors can be set to adjust the positive or negative impact of different data features on the final fusion result. For example, if a certain indicator deviates significantly from the normal range, the corresponding data weighting can be reduced to reduce the impact of outliers. Complex exponential transformation: Complex exponential transformation is applied to motion modal data to enhance the dynamic features in the motion modal data, so that the changes in motion state can be better reflected in the fused data.

[0121] The embodiment of the present invention adopts the design of multimodal data fusion by introducing weights, penalty factors and distribution characteristics to ensure that the fusion result can effectively reflect the patient's recovery status and provide support for subsequent status monitoring and analysis.

[0122] In another embodiment, after performing data fusion on the key modal feature data of each modal data to obtain fused multimodal data, the method further includes:

[0123] Formula (4) is used to perform data enhancement processing on the fused multimodal data:

[0124] Formula (4),

[0125] in, represents the fused multimodal data, represents the enhanced fusion data, Represents the fused multimodal data The frequency, Represents the fused multimodal data The phase offset, Represents the fused multimodal data The variance of Represents the fused multimodal data The mean of .

[0126] In this embodiment, data enhancement is used to introduce frequency The oscillation function of , makes data enhancement not limited to simple scale changes. This oscillation can perturb the data at different frequencies, increasing the diversity and complexity of the data. In addition, data enhancement includes phase bias , which makes the data generated have certain phase rotation characteristics. This enhances the model's adaptability to data changes and helps improve its performance when dealing with rapidly changing or periodic features. It provides information from different sources, and by enhancing this information, the model's ability to capture a variety of input characteristics can be further improved.

[0127] Finally, the processor inputs the fused multimodal data into the rehabilitation detection calculation model to obtain the rehabilitation detection results of the user to be rehabilitated. The specific processing method includes:

[0128] Formula (5) is used to calculate the multimodal data to obtain the rehabilitation test results of the user to be rehabilitated:

[0129] Formula (5),

[0130] Among them, C represents the rehabilitation status detection index, which represents the patient's rehabilitation status score. represents the enhanced fused multimodal data, represents the preprocessed physiological modality data, represents the preprocessed motion modal data, Represents the fused multimodal data The impact factor, Represents the preprocessed physiological modality data and preprocessed motion modal data The attenuation factor, Represents the recovery status threshold.

[0131] if Indicates good condition, continue to maintain training intensity, if It is recommended to reduce the intensity of training and take rest. Based on the test results, personalized feedback and suggestions can be provided to help patients adjust their training methods and intensity.

[0132] In another embodiment of the present invention, referring to Figure 2 The present application also provides a flowchart of a training method for a rehabilitation detection calculation model in a pelvic floor rehabilitation training state detection method based on multimodal data fusion, wherein the training method comprises:

[0133] Step 110, obtaining multimodal fusion data of the user's historical rehabilitation training process;

[0134] Step 120, inputting the multimodal fusion data into the rehabilitation detection calculation model;

[0135] Step 130, obtaining the detection result output by the model;

[0136] Step 140, calculating the deviation between the output detection result and the actual result;

[0137] Step 150, adjusting the parameters of the rehabilitation detection calculation model according to the deviation value;

[0138] Steps 110 to 150 are repeated until the deviation between the test result obtained by the rehabilitation test calculation model and the actual result is within a preset range.

[0139] The method of this embodiment is intended to calculate the pelvic floor detection results through model calculation using an artificial intelligence algorithm, thereby improving the intelligence and efficiency of the calculation.

[0140] Figure 3 This is a structural block diagram of a pelvic floor rehabilitation training state detection system based on multimodal data fusion provided in an embodiment of the present application. Figure 3 As shown, a pelvic floor rehabilitation training state detection system based on multimodal data fusion provided by an embodiment of the present application may include:

[0141] Memory 310, configured to store instructions; and

[0142] The processor 320 is configured to call instructions from the memory 310 and implement the above-mentioned pelvic floor rehabilitation training state detection method based on multimodal data fusion when executing the instructions.

[0143] An embodiment of the present application also provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute the above-mentioned pelvic floor rehabilitation training state detection method based on multimodal data fusion.

[0144] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0145] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0146] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0147] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0148] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0149] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0150] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0151] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0152] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A pelvic floor rehabilitation training status detection method based on multimodal data fusion, characterized in that: include: Acquiring multimodal data of the user to be rehabilitated collected during rehabilitation training, wherein the multimodal data includes motion modal data and physiological modal data; Extracting key information of the multimodal data to obtain key modal feature data of each modal data in the multimodal data; Performing data fusion on the key modal feature data of each modal data to obtain fused multimodal data; The fused multimodal data is input into the rehabilitation detection calculation model to obtain the rehabilitation detection results of the user to be rehabilitated. Wherein, before extracting the key information of the multimodal data, the pelvic floor rehabilitation training state detection method based on multimodal data fusion further includes: The physiological modality data is preprocessed using formula (1): Formula (1) in, represents the preprocessed physiological modality data, represents the initial physiological modality data, Represents the initial physiological modality data The mean of Represents the initial physiological modality data The cut-off frequency, Represents the initial physiological modality data The average frequency of Represents the initial physiological modality data The standard deviation of .

2. The pelvic floor rehabilitation training state detection method based on multimodal data fusion according to claim 1, characterized in that: Extracting key information of the multimodal data to obtain key modal feature data of each modal data in the multimodal data includes: acquiring an electromyographic signal from the physiological modality data; performing a preset process on the electromyographic signal to extract a characteristic frequency therefrom; Performing preset processing on the amplitude value of the electromyography signal, and determining the amplitude value of the electromyography signal after the preset processing as the muscle contraction intensity index value; The characteristic frequency and the muscle contraction strength index value are determined as key modal feature data of the physiological modal data.

3. The pelvic floor rehabilitation training state detection method based on multimodal data fusion according to claim 2, characterized in that: Extracting key information of the multimodal data to obtain key modal feature data of each modal data in the multimodal data also includes: Extracting speed data and acceleration data of the user to be rehabilitated from the motion modality data; Calculating a stability evaluation value according to the rate of change of the acceleration data; The velocity data, the acceleration data, and the stability evaluation value are determined as key modal feature data of the motion modal data.

4. The pelvic floor rehabilitation training state detection method based on multimodal data fusion according to claim 1, characterized in that: Before extracting the key information of the multimodal data, the pelvic floor rehabilitation training state detection method based on multimodal data fusion further includes: The motion modal data is preprocessed using formula (2): Formula (2) in, represents the preprocessed motion modal data, represents the initial motion modal data, Represents the initial motion modal data The mean of Represents the initial motion modal data The cut-off frequency, Represents the initial motion modal data The average frequency of Represents the initial motion modal data The standard deviation of Represents the initial motion modal data The Kalman gain.

5. The pelvic floor rehabilitation training state detection method based on multimodal data fusion according to claim 4, characterized in that: The key modal feature data of each modal data is subjected to data fusion to obtain fused multimodal data, including: Processing the key modal feature data so that the key modal feature data of the physiological modal data and the key modal data of the motion modal data meet data fusion requirements; The key modality data of the physiological modality data and the key modality data of the motion modality data are fused according to formula (3): Formula (3) in, Represents the fused multimodal data, which combines the information of physiological modality data and motion modality data. represents the preprocessed physiological modality data, represents the preprocessed motion modal data, Represents the preprocessed physiological modality data The weight adjustment factor is Represents the preprocessed motion modal data The weight adjustment factor is represents the Laplace distribution density function, j represents the preprocessed motion modal data The scale factor is used to control the data expansion and contraction in the data fusion process. a represents the preprocessed physiological modality data. The penalty factor is i, i represents the preprocessed motion modal data The penalty factor is t, which represents the initial physiological modality data. and initial motion modal data The data collection time point, k represents the preprocessed physiological modality data The scaling factor of .

6. The pelvic floor rehabilitation training state detection method based on multimodal data fusion according to claim 5, characterized in that: After the key modal feature data of each modal data is fused to obtain fused multimodal data, the pelvic floor rehabilitation training state detection method based on multimodal data fusion further includes: Formula (4) is used to perform data enhancement processing on the fused multimodal data: Formula (4) in, represents the fused multimodal data, represents the enhanced fusion data, Represents the fused multimodal data The frequency, Represents the fused multimodal data The phase offset, Represents the fused multimodal data The variance of Represents the fused multimodal data The mean of .

7. The pelvic floor rehabilitation training state detection method based on multimodal data fusion according to claim 6, characterized in that: The fused multimodal data is input into the rehabilitation detection calculation model to obtain the rehabilitation detection results of the user to be rehabilitated, including: The rehabilitation detection calculation model uses formula (5) to calculate the multimodal data to obtain the rehabilitation detection result of the user to be rehabilitated: Formula (5) Among them, C represents the rehabilitation status detection index, which represents the patient's rehabilitation status score. represents the enhanced fused multimodal data, represents the preprocessed physiological modality data, represents the preprocessed motion modal data, Represents the fused multimodal data The impact factor, Represents the preprocessed physiological modality data and preprocessed motion modal data The attenuation factor, Represents the recovery status threshold.

8. The pelvic floor rehabilitation training status detection system based on multimodal data fusion is characterized by: include: A memory storing instructions; as well as A processor, wherein the processor executes the instruction to implement the pelvic floor rehabilitation training state detection method based on multimodal data fusion according to any one of claims 1-7.

Images