Self-adaptive electrical stimulation pelvic floor treatment system combined with reinforcement learning and method thereof

Abstract

The invention relates to the technical field of self-adaptive electrical stimulation pelvic floor treatment systems, in particular to a self-adaptive electrical stimulation pelvic floor treatment system combined with reinforcement learning and a control method of the self-adaptive electrical stimulation pelvic floor treatment system combined with reinforcement learning, and the self-adaptive electrical stimulation pelvic floor treatment system comprises a micro electrical stimulation device which is used for applying electrical stimulation to pelvic floor muscles; the multi-mode biological feedback sensor array is used for collecting physiological signals of a patient; the data processing module is connected with the multi-mode biological feedback sensor array and used for processing the collected physiological signals; the reinforcement learning controller is connected with the data processing module and the micro electrical stimulation device and is used for dynamically adjusting electrical stimulation parameters based on the processed physiological signals; the human-computer interaction interface is connected with the reinforcement learning controller and is used for receiving doctor input and displaying treatment information; through real-time self-adaptive control and personalized parameter adjustment, a target muscle group can be stimulated more accurately, so that muscles are stimulated in an optimal state all the time, and the pelvic floor muscle strength improvement degree of a patient reaches 68.5% and is far higher than that of a traditional method.

Description

Technical Field

[0001] The present invention relates to the technical field of adaptive electrical stimulation pelvic floor therapy systems, and more specifically, to adaptive electrical stimulation pelvic floor therapy systems and methods thereof combined with reinforcement learning. Background Art

[0002] Pelvic floor dysfunction is a common gynecological disease that seriously affects the quality of life of patients. In recent years, electrical stimulation therapy has been widely used in the treatment of pelvic floor dysfunction as a non-invasive treatment method. However, traditional electrical stimulation treatment methods have many limitations and are difficult to meet the growing demand for personalized treatment.

[0003] Early pelvic floor electrical stimulation therapies mostly used fixed parameter schemes. For example, Beicheng Medical Technology (Jiangsu) Co., Ltd. proposed a pelvic floor feedback therapy system in its patent application for pelvic floor feedback therapy system, control method and stimulation unit thereof (application number: 202410693656.X). Although the system introduces the concept of sub-units and realizes the division of stimulation areas, its parameter adjustment still mainly relies on preset schemes and doctor experience, and lacks real-time adaptive capabilities. This method is difficult to adapt to the individual differences of different patients and the changes in the status of the same patient during the treatment process, which can easily lead to unstable treatment effects and may sometimes even aggravate symptoms.

[0004] Subsequently, some researchers tried to introduce a simple feedback mechanism to adjust the electrical stimulation parameters. This type of method is usually adjusted based on the patient's subjective feedback or a single physiological indicator. Although it is an improvement over the fixed parameter scheme, there are still many problems. First, the patient's subjective feedback may not be accurate or timely; second, a single physiological indicator is difficult to fully reflect the complex state of the pelvic floor muscles; third, this simple feedback adjustment lacks systematicity and foresight, and cannot achieve long-term treatment optimization.

[0005] In recent years, with the development of artificial intelligence technology, some researchers have begun to try to apply machine learning algorithms to pelvic floor electrical stimulation therapy. However, most of these attempts remain at the level of offline analysis and simple parameter prediction, and cannot achieve real-time adaptive control. In addition, existing intelligent solutions often ignore the long-term effects of treatment and lack overall control over the patient's recovery process.

[0006] In view of the shortcomings of existing technologies, there is an urgent need for a pelvic floor electrical stimulation therapy system and method that can achieve precise control, real-time adaptation, and long-term optimization capabilities. Summary of the invention

[0007] The present invention is proposed in this context. To address the above problems, the present invention proposes an adaptive electrical stimulation pelvic floor treatment system and method combined with reinforcement learning, which solves the problems of fixed parameters and untimely adjustment in traditional methods, and overcomes the defects of simple feedback mechanism that lacks systematicity and foresight.

[0008] The present invention provides an adaptive electrical stimulation pelvic floor treatment system combined with reinforcement learning, comprising: a micro-electric stimulation device for applying electrical stimulation to the pelvic floor muscles; a multimodal biofeedback sensor array for collecting physiological signals of patients; a data processing module connected to the multimodal biofeedback sensor array for processing the collected physiological signals; a reinforcement learning controller connected to the data processing module and the micro-electric stimulation device for dynamically adjusting electrical stimulation parameters based on the processed physiological signals; a human-computer interaction interface connected to the reinforcement learning controller for receiving doctor input and displaying treatment information; a data storage management module connected to the data processing module, the reinforcement learning controller and the human-computer interaction interface for storing and managing treatment data.

[0009] Specifically, the micro-electrical stimulation device includes: a multi-channel stimulation electrode array for contacting the pelvic floor muscles; a digital programmable waveform generator for generating an electrical stimulation waveform; a high-precision current source connected to the digital programmable waveform generator for converting the waveform into a current output; and an impedance detection circuit connected to the multi-channel stimulation electrode array for monitoring the electrode-tissue interface impedance.

[0010] Specifically, the multimodal biofeedback sensor array includes: a surface electromyography sensor array for collecting electromyographic signals; a pressure sensor for detecting pelvic floor muscle pressure; an acceleration sensor for detecting patient posture; and a temperature sensor for monitoring local temperature changes.

[0011] Specifically, the data processing module includes: a signal preprocessing unit for filtering and denoising the original signal; a feature extraction unit for extracting features from the preprocessed signal; and a data fusion unit for integrating multimodal data to generate a comprehensive state representation.

[0012] Specifically, the reinforcement learning controller includes:

[0013] A state evaluation module is used to evaluate the current pelvic floor muscle state; an Actor-Critic network is used to generate optimal stimulation parameters; and an experience playback buffer is used to store historical experience data; wherein the Actor-Critic network is trained based on a reward function of multi-objective optimization.

[0014] Specifically, the reinforcement learning controller also includes an adaptive noise injection mechanism for balancing exploration and exploitation.

[0015] Specifically, it also includes a long-term optimization strategy generator, which includes: a multi-scale time series modeling module for predicting future state trends; and a Monte Carlo tree search module for generating long-term treatment strategies.

[0016] Specifically, the human-computer interaction interface includes a doctor operation interface for setting treatment parameters and viewing analysis reports; a patient feedback interface for collecting patients' subjective feelings; and a data visualization module for displaying real-time treatment data and long-term rehabilitation progress.

[0017] Specifically, the data storage management module includes: a local database for storing recent treatment data; a cloud data center for long-term data storage and large-scale analysis; and a data security module for ensuring the security of data transmission and storage.

[0018] The pelvic floor treatment method using the system includes the following steps: obtaining the basic information and initial state of the patient; determining the initial electrical stimulation parameters based on the basic information and initial state; performing electrical stimulation treatment on the patient using the micro-electrical stimulation device; collecting the patient's real-time physiological signals through the multimodal biofeedback sensor array; processing and extracting features of the collected physiological signals using the data processing module; dynamically adjusting the electrical stimulation parameters based on the processed signals using the reinforcement learning controller; repeating the electrical stimulation, signal collection and parameter adjustment steps until the preset treatment time is reached; and generating a long-term treatment plan based on historical treatment data using the long-term optimization strategy generator.

[0019] The core of the present invention is the reinforcement learning controller, which can adjust the electrical stimulation parameters in real time based on multimodal biofeedback data to achieve personalized treatment. At the same time, the system also has long-term optimization capabilities and can continuously adjust the treatment strategy according to the patient's recovery process. This design not only solves the problem of fixed parameters and untimely adjustment in traditional methods, but also overcomes the defects of simple feedback mechanisms that lack systematicity and foresight.

[0020] The beneficial effects of the present invention are mainly reflected in the following aspects:

[0021] First, the treatment effect is significantly improved. Through real-time adaptive control and personalized parameter adjustment, this system can stimulate the target muscle groups more accurately, so that the muscles are always stimulated in the best state. The experimental results show that the improvement of pelvic floor muscle strength of patients using this system reaches 68.5%, which is much higher than traditional methods.

[0022] Secondly, the patient experience has been greatly improved. The system's adaptive characteristics and humanized interface design have significantly improved the comfort of treatment and patient compliance. In the experiment, the treatment compliance of patients using this system was as high as 94.2%, far exceeding other solutions.

[0023] Furthermore, safety is guaranteed. Multimodal biofeedback and real-time status assessment mechanisms can detect and prevent potential risks in a timely manner, reducing the incidence of adverse reactions to 3.3%, greatly improving the safety of treatment.

[0024] Finally, the system has long-term optimization capabilities. Through continuous learning and adjustment, the system can develop the best long-term treatment plan for each patient and achieve continuous improvement in treatment effects.

[0025] In summary, the adaptive electrical stimulation pelvic floor treatment system of the present invention not only solves many problems existing in the prior art, but also makes significant progress in treatment effect, patient experience and safety. It provides a new solution for the treatment of pelvic floor dysfunction, which is expected to greatly improve the clinical treatment effect and improve the quality of life of patients. It has important clinical application value and broad market prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is the overall logic block diagram of the system of the present invention.

[0027] Figure 2 It is a logic block diagram of the micro-electric stimulation device of the present invention.

[0028] Figure 3 It is a logic block diagram of the multimodal biofeedback sensor array of the present invention.

[0029] Figure 4 It is a logic block diagram of the data processing module of the present invention.

[0030] Figure 5 This is a logic block diagram of the reinforcement learning controller of the present invention.

[0031] Figure 6 It is a schematic diagram of the system structure of the present invention.

[0032] Figure 7 It is a schematic diagram of the structure of the micro-electric stimulation device of the present invention.

[0033] Figure 8 Schematic diagram of the structure of the multimodal biofeedback sensor array of the present invention.

[0034] Fig. 9 It is a structural schematic diagram of the data processing module of the present invention.

[0035] Fig.10 It is a structural schematic diagram of the reinforcement learning controller of the present invention.

[0036] Fig.11 It is a logic block diagram of the human-computer interaction interface of the present invention.

[0037] Fig.12 It is a logical block diagram of the data storage management module of the present invention. DETAILED DESCRIPTION

[0038] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects thereof are described in detail below in conjunction with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0039] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0040] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0041] The present invention relates to an adaptive electrical stimulation pelvic floor treatment system combined with reinforcement learning and a method thereof, belonging to the technical field of medical devices, and in particular to a pelvic floor dysfunction treatment technology.

[0042] like Figure 1-12 As shown, the adaptive electrical stimulation pelvic floor treatment system of the present invention comprises a micro-electrical stimulation device 1, a multimodal biofeedback sensor array 2, a data processing module 3, a reinforcement learning controller 4, a human-computer interaction interface 5, and a data storage management module 6. These modules work together to form a closed-loop control system to achieve accurate electrical stimulation and real-time monitoring of the pelvic floor muscles.

[0043] Please refer to Figure 7 The micro-electric stimulation device 1 is the execution unit of the system, which is responsible for applying electrical stimulation to the pelvic floor muscles. It includes a multi-channel stimulation electrode array 11, a digital programmable waveform generator 12, a high-precision current source 13, and an impedance detection circuit 14. The multi-channel stimulation electrode array 11 adopts an 8x8 matrix layout, with a total of 64 independently controllable electrodes, each with a diameter of 2mm, a spacing of 3mm, and a total area of ​​about 25x25mm 2. This design can cover a sufficiently large pelvic floor muscle area while ensuring the accuracy of stimulation. The digital programmable waveform generator 12 is capable of generating a variety of waveforms, including square waves, sine waves, triangle waves, and exponential decay waves, with a frequency range of 1-200Hz, a step of 0.1Hz, and a pulse width of 50-500μs, a step of 10μs. This flexible waveform generation capability allows the system to adapt to the needs of different patients. The output range of the high-precision current source 13 is 0-50mA, the resolution is 0.1mA, the dynamic range is greater than 60dB, and the output impedance is greater than 1MΩ at a frequency of 1kHz. This high-precision design ensures precise control of the stimulation intensity. The measurement range of the impedance detection circuit 14 is 100Ω-10kΩ, the accuracy is ±5%, and the measurement frequency is 1kHz. By real-time monitoring of the electrode-tissue interface impedance, the system can promptly detect problems such as electrode detachment or poor contact to ensure the safety and effectiveness of treatment.

[0044] Please refer to Figure 8 , the multimodal biofeedback sensor array 2 is the sensing unit of the system, which is used to collect multiple physiological signals of the patient. It includes a surface electromyography (sEMG) sensor array 21, a pressure sensor 22, an acceleration sensor 23 and a temperature sensor 24. The sEMG sensor array 21 adopts an 8x8 matrix layout, which is staggered with the stimulation electrodes, uses Ag / AgCl dry electrodes, has a sampling rate of 2000Hz, a resolution of 24-bit ADC, and a common mode rejection ratio greater than 100dB. This high-precision electromyographic signal acquisition can accurately reflect the contraction state of the pelvic floor muscles. The pressure sensor 22 adopts a capacitive design, with a total of 16, distributed in key positions, with a measurement range of 0-100mmHg, a resolution of 0.1mmHg, and a response time of less than 5ms. These pressure sensors can detect the contraction strength and duration of the pelvic floor muscles. The acceleration sensor 23 is a three-axis design with a range of ±2g, a resolution of 16 bits, and a sampling rate of 100Hz, which is used to detect changes in the patient's posture. The temperature sensor 24 uses a thermistor with a measurement range of 30-42°C, an accuracy of ±0.1°C, and a response time of less than 1s. It is used to monitor local temperature changes and prevent tissue damage caused by excessive stimulation.

[0045] Please refer to Fig. 9The data processing module 3 is responsible for processing and analyzing the collected original physiological signals. It includes a signal preprocessing unit 31, a feature extraction unit 32 and a data fusion unit 33. The signal preprocessing unit 31 first performs bandpass filtering on the original signal, using a 4th-order Butterworth filter with a passband of 10-500Hz (for sEMG), and setting a 50 / 60Hz notch to suppress power frequency interference. Then, the wavelet transform based on Daubechies 4 wavelet is used for denoising, with a decomposition level of 5 and an adaptive threshold selection method. Finally, the least mean square error (LMS) adaptive filtering algorithm is applied to remove motion artifacts, with a filter length of 32 orders, an initial step size parameter μ of 0.01, and a convergence condition of a mean square error less than 0.001 or more than 1000 iterations. The feature extraction unit 32 calculates time domain features (such as RMS, mean absolute value, waveform length, etc.), frequency domain features (such as power spectral density, median frequency, etc.) and time-frequency domain features (such as wavelet packet decomposition coefficients). The data fusion unit 33 uses Kalman filtering to perform multi-sensor data fusion. The state vector includes [electromyography intensity, pressure, posture angle, temperature]. The observation noise covariance R is a diagonal matrix. According to the accuracy setting of each sensor, the process noise covariance Q is dynamically adjusted using an adaptive estimation method. At the same time, the Dempster-Shafer evidence theory is applied to deal with uncertainty. The framework is identified as {normal, mild abnormality, moderate abnormality, severe abnormality}, and the Yager combination rule is used to deal with highly conflicting evidence.

[0046] Please refer to Fig.10 , the reinforcement learning controller 4 is the core decision-making unit of the system, which is responsible for dynamically adjusting the electrical stimulation parameters according to the processed physiological signals. It includes a state evaluation module 41, an Actor-Critic network 42 and an experience replay buffer 43. The state evaluation module 41 combines the fuzzy logic reasoning system and the support vector machine (SVM) classifier to evaluate the current pelvic floor muscle state. The input variables of the fuzzy logic system include myoelectric intensity, pressure, posture angle, temperature and patient feedback. Each variable defines 3-5 fuzzy sets. The rule base contains 50-100 expert-defined IF-THEN rules, which are defuzzified using the Mamdani reasoning method and the center of gravity method. The SVM classifier uses a radial basis function (RBF) kernel, a penalty factor C of 10, and a γ parameter of 0.1. These parameters are obtained by cross-validation and grid search optimization. The Actor-Critic network 42 uses an improved deep deterministic policy gradient (DDPG) algorithm. The actor network μ(s|θ μ ) and Critic network Q(s,a|θ Q ) all use a fully connected neural network structure, with a hidden layer configuration of [64,128,64] and an activation function of ReLU. In order to balance exploration and utilization, an adaptive noise injection mechanism is introduced:

[0047] action=μ(s t )+N(0,σ(t))

[0048] σ(t)=max(σ min ,σ0·exp(-βt)+α|Q(s t+1 ,μ(s t+1 ))-Q(s t ,μ(s t ))|)

[0049] Among them, σ min is the minimum noise level, set to 0.01; σ0 is the initial noise level, set to 0.5; β is the attenuation rate, set to 0.001; α is the Q value difference influence factor, set to 0.1.

[0050] This adaptive noise mechanism can maintain a high degree of exploration in the early stage of training, gradually reduce the exploration as learning progresses, and dynamically adjust the noise level according to the change of Q value, which helps to find the optimal stimulation strategy in the complex pelvic floor muscle state space.

[0051] In order to make better use of historical experience, the present invention adopts a hierarchical experience playback strategy. The experience playback buffer 43 includes a short-term buffer and a long-term buffer. The short-term buffer has a capacity of 10,000 samples and stores the experience of the most recent N treatments; the long-term buffer has a capacity of 100,000 samples and stores representative historical experience. When sampling, the probability of sampling from the short-term buffer is 70%, and the probability of sampling from the long-term buffer is 30%. The priority sampling strategy sets the sampling priority based on the absolute value of the TD error:

[0052]

[0053] Among them, δ i is the TD error, ε is a small positive number (set to 0.01), and α is the priority exponent (set to 0.6).

[0054] This hierarchical experience replay strategy can balance recent experience and long-term experience, improving learning efficiency and generalization ability.

[0055] One of the innovations of this invention is the use of a multi-objective optimization reward function that takes into account treatment efficacy, patient comfort, energy consumption, and muscle fatigue:

[0056] R=w1·R effect +w2·R comfort -w3·R energy -w4·R fatigue

[0057] Among them, Reffect Calculate the therapeutic effect reward based on the strength and duration of the electromyographic signal; R comfort Comfort rating based on patient feedback (0-10 points); R energy Calculate the penalty term based on the output energy; R fatigue The fatigue penalty term is calculated based on the continuous stimulation time. The initial weight values ​​are set to w1 = 0.5, w2 = 0.3, w3 = 0.1, and w4 = 0.1, which can be adjusted dynamically according to the progress of treatment. This multi-objective reward function design can improve the treatment effect while taking into account patient comfort and system energy efficiency to avoid excessive muscle fatigue.

[0058] In order to accelerate the learning process of new patients, this paper introduces a meta-learning-based strategy transfer method. Specifically, the Model-Agnostic Meta-Learning (MAML) algorithm is used, the inner loop learning rate α is set to 0.01, and the outer loop learning rate β is set to 0.001. Meta-training tasks are randomly selected from historical patient data, and 5 tasks are used for each meta-update. For new patients, the initial strategy parameters are obtained in the following way:

[0059]

[0060] Among them, θ meta is the initial parameter obtained by meta-learning, and L(θ) is the loss function for new patients. This method can quickly adapt to the characteristics of new patients and significantly reduce the initial learning time.

[0061] Please refer to Fig.11 The human-computer interaction interface 5 is a platform for information exchange between the system and doctors and patients. It includes a doctor operation interface 51, a patient feedback interface 52 and a data visualization module 53. The doctor operation interface 51 allows doctors to set initial treatment parameters, monitor the treatment process and adjust treatment strategies. For example, doctors can set safety thresholds for electrical stimulation, such as a maximum stimulation intensity of no more than 40 mA and a maximum duration of no more than 30 minutes. The patient feedback interface 52 is used to collect patients' subjective feelings, including a comfort scoring system (0-10 scale, recorded every 30 seconds) and feedback on symptom improvement (assessed using a 5-level Likert scale). The data visualization module 53 displays multi-channel electromyography, pressure distribution heat map and 3D muscle activation model in real time, with an update frequency of 2Hz. This intuitive visual display helps doctors quickly evaluate the treatment effect and make timely adjustments.

[0062] Please refer to Fig.12, the data storage management module 6 is responsible for the storage, synchronization and security management of data. It includes a local database 61, a cloud data center 62 and a data security module 63. The local database 61 uses a SQLite relational database to store recent treatment data and system configuration. The data table structure includes a patient basic information table, a treatment session table, a real-time data table (minute-level summary), a daily summary table and an equipment configuration table. In order to optimize query performance, indexes are established for frequently accessed fields. The cloud data center 62 uses the Hadoop distributed file system (HDFS) to store long-term data, the data replication factor is set to 3 to ensure reliability, and the block size is set to 128MB to optimize large file storage. Apache Spark is used for large-scale data processing and analysis, including data cleaning, feature engineering and distributed machine learning model training. The data security module 63 adopts multi-level security measures, including TLS1.3 protocol for transmission encryption, AES-256-GCM algorithm for storage encryption, role-based access control (RBAC) and multi-factor authentication. In addition, blockchain technology is used to ensure the integrity and immutability of audit logs.

[0063] The pelvic floor treatment method of the present invention comprises the following steps:

[0064] Step S1, system initialization. First, execute the device self-test program to check the functions of all hardware components and calibrate the sensor array. Then, the doctor enters the patient's basic information through the operation interface, and the system automatically obtains historical data (if any) from the electronic medical record system. Next, obtain the latest pre-trained reinforcement learning model from the cloud, and load it if there are patient-specific model parameters. Finally, initialize the meta-learning module to prepare for rapid adaptation to new patients.

[0065] Step S2, pre-treatment assessment. First, the patient fills out a questionnaire on current symptoms, including frequency of urinary incontinence, degree of constipation, etc. Then, the system performs a baseline electromyography measurement for 30 seconds. During the measurement, the patient is required to maintain a relaxed state and a maximum voluntary contraction state for 15 seconds each. Next, AI analyzes the baseline data and generates initial treatment recommendations, including recommended stimulation frequency, intensity, and duration.

[0066] Step S3, parameter setting. The doctor reviews the initial treatment recommendations generated by AI and makes necessary adjustments based on clinical experience. For example, if the patient has a pacemaker, the doctor may limit the maximum stimulation frequency to below 50Hz. The system performs a parameter safety check to ensure that all parameters are within the preset safety range. For example, the stimulation intensity must not exceed 50mA, and the single treatment time must not exceed 45 minutes. Finally, the finalized parameters are transmitted to the micro-electrical stimulation device 1.

[0067] Step S4, electrical stimulation is applied. The reinforcement learning controller 4 outputs stimulation parameters according to the initial settings. The micro-electrical stimulation device 1 starts to perform stimulation, gradually increasing from a low intensity (usually 1 mA) by 0.5 mA each time until the set intensity or the patient-reported sensation threshold is reached. During this process, the impedance detection circuit 14 monitors the electrode-tissue interface impedance in real time. If an impedance abnormality is detected (for example, the impedance suddenly rises to more than 5 kΩ), the system will immediately suspend the stimulation and notify the doctor to prevent possible skin irritation or electrode detachment.

[0068] Step S5, physiological signal acquisition. The multimodal biofeedback sensor array 2 continuously acquires signals at a sampling rate of 2000 Hz. The surface electromyography sensor array 21 records the electrical activity of the pelvic floor muscles, the pressure sensor 22 monitors the strength of muscle contraction, the acceleration sensor 23 detects changes in the patient's posture, and the temperature sensor 24 monitors local temperature changes. These data are transmitted to the embedded controller in real time via the SPI interface. The controller performs preliminary data verification, such as checking whether the signal amplitude is within the normal range (usually the electromyography signal amplitude should be between 10μV and 1mV), and adding an accurate timestamp (with an accuracy better than 1ms).

[0069] Step S6, data processing and feature extraction. The data processing module 3 first preprocesses the original signal. For the electromyographic signal, a 4th-order Butterworth bandpass filter (10-500Hz) is applied, and a 50 / 60Hz notch filter is used to remove power frequency interference. Then, a wavelet transform based on Daubechies 4 wavelets is used for denoising, the decomposition level is 5, and the VisuShrink method is used to select the threshold. Next, the feature extraction unit 32 calculates a series of features, including:

[0070] 1) Time domain characteristics: RMS value, mean absolute value (MAV), waveform length (WL), zero crossing rate (ZC, threshold set to 10μV).

[0071] 2) Frequency domain features: The power spectral density (PSD) was calculated using the Welch method with a window length of 256 ms and an overlap rate of 50%. The median frequency (MDF) and mean power frequency (MPF) were extracted.

[0072] 3) Time-frequency domain characteristics: Use Daubechies 4 wavelet to perform 6-level wavelet packet decomposition and calculate the energy proportion of each frequency band and the maximum energy ratio.

[0073] Finally, the data fusion unit 33 uses a Kalman filter to fuse the multi-sensor data. The state vector is defined as [EMG RMS, pressure, posture angle x, posture angle y, posture angle z, temperature]. The observation noise covariance R is set according to the accuracy of each sensor. For example, the measurement error of EMG RMS is set to 5%, the pressure is 0.5 mmHg, the posture angle is 0.1°, and the temperature is 0.1°C. The process noise covariance Q adopts an adaptive estimation method, and the initial value is set to a diagonal matrix with diagonal elements of [0.01, 0.1, 0.01, 0.01, 0.001].

[0074] Step S7, real-time state evaluation. The state evaluation module 41 combines the fuzzy logic system and the SVM classifier to evaluate the current pelvic floor muscle state. The input variables of the fuzzy logic system include standardized electromyographic RMS, pressure, posture change amplitude and temperature change. Each variable defines 5 fuzzy sets (extremely low, low, medium, high, and extremely high). The rule base contains 80 expert-defined IF-THEN rules, for example: IF electromyographic RMS is high AND pressure is medium AND posture change is low AND temperature change is low THEN muscle state is good. Mamdani reasoning method and center of gravity method are used for defuzzification. The SVM classifier uses the RBF kernel and optimizes the optimal parameters C=10 and γ=0.1 through 5-fold cross validation. The classification results include four categories: normal, mildly abnormal, moderately abnormal, and severely abnormal. The final state evaluation result is obtained by weighting the fuzzy logic and SVM results, with a weight ratio of 0.6:0.4.

[0075] Step S8, dynamic adjustment of parameters. The Actor-Critic network 42 of the reinforcement learning controller 4 selects the optimal stimulation parameters based on the current state evaluation results. The Actor network outputs normalized stimulation parameters, including frequency (1-200Hz), intensity (0-50mA), pulse width (50-500μs) and electrode selection (switching state of 64 electrodes). In order to ensure exploration, an adaptive noise injection mechanism is applied. For example, if the current Q value changes greatly (|ΔQ|>0.5), the noise level is increased to facilitate exploration. The new stimulation parameters are applied to the micro-electrical stimulation device 1 after safety checks. The safety check includes: ensuring that the frequency does not exceed 200Hz, the intensity does not exceed 50mA, the pulse width does not exceed 500μs, the duration of a single stimulation does not exceed 5 seconds, and the rest time is not less than 1.5 times the stimulation time.

[0076] Step S9, patient feedback collection. The patient feedback interface 52 prompts the patient to update the comfort score (0-10 points) every 30 seconds. If the score is less than 3 points, the system will immediately reduce the stimulation intensity by 20%. In addition, the patient can report any discomfort or abnormal sensations, such as tingling, burning, etc. at any time. These subjective feedbacks will be integrated into the state assessment process and affect the next parameter adjustment decision.

[0077] Step S10, post-treatment evaluation. After the treatment, the post-treatment electromyography measurement is immediately performed and compared with the baseline. The following indicators are calculated: the percentage increase in the maximum voluntary contraction force (MVC), the improvement in rapid contraction ability (by calculating the average RMS value of 10 1-second contractions), and the improvement in sustained contraction ability (by calculating the fatigue index of 60-second sustained contractions). The patient completes the symptom improvement assessment questionnaire to evaluate the degree of reduction in urinary incontinence frequency, the degree of improvement in constipation symptoms, etc. The system generates a single treatment effect report based on these objective and subjective indicators.

[0078] Step S11, model update. Based on the treatment effect, patient feedback and energy consumption, the comprehensive reward value is calculated:

[0079] R = 0.5*(MVC improvement% / 20%) + 0.3*(comfort score / 10) - 0.1*(energy consumption / maximum allowed energy consumption) - 0.1*(total stimulation time / maximum allowed time)

[0080] Among them, when the MVC is improved by 20%, the comfort score is 10 points, the energy consumption does not exceed the set maximum value, and the total stimulation time does not exceed the maximum allowed time, the score of each item is 1. This multi-objective reward function design aims to balance the treatment effect, patient experience and system efficiency.

[0081] Next, update the reinforcement learning model. First, the new experience data (s t ,a t ,r t ,s t+1 ) is stored in the hierarchical experience replay buffer 43. A small batch (batch size = 64) is sampled from the buffer, of which 70% comes from the short-term buffer and 30% comes from the long-term buffer. Then, the Critic network is updated:

[0082]

[0083] y i =r i +γ·Q′(s i+1 ,μ′(s i+1 |θ μ ′)|θ Q ′)

[0084] Among them, θ Q is the Critic network parameter, θ′ Q is the target Critic network parameter, and γ is the discount factor (set to 0.99). Update the Actor network:

[0085]

[0086] Finally, soft update the target network:

[0087] θ′=τθ+(1-τ)θ′

[0088] Where τ is the soft update rate, which is set to 0.001. This soft update mechanism can stabilize the learning process and avoid drastic fluctuations in value estimates.

[0089] Step S12, long-term optimization strategy generation. The long-term optimization strategy generator 7 first extracts the treatment data of the last 30 days from the local database 61, including daily treatment parameters, effect evaluation and patient feedback. The data is converted into a format suitable for the LSTM network, namely [date, frequency, intensity, pulse width, electrode selection, MVC improvement%, comfort score, symptom improvement].

[0090] The multi-scale time series modeling module uses an LSTM network to predict the state and optimal parameters for the next 7 days. The LSTM network structure is 3 layers, with 64, 128, and 64 units in each layer, and the dropout rate is set to 0.2. The input is the daily treatment data for the past 30 days, with 8 time steps per day (corresponding to 8 treatments). The output is the predicted state and recommended parameters for the next 7 days. At the same time, the prediction confidence interval is generated, using the Monte Carlo dropout method, performing 100 forward propagations, and taking the 2.5% and 97.5% quantiles as the 95% confidence interval.

[0091] Next, the Monte Carlo Tree Search (MCTS) module uses the LSTM prediction results as prior knowledge to simulate the treatment process for the next 14 days. Each node of the search tree represents the treatment status of a day, and each branch represents a possible parameter selection. The UCT (Upper Confidence Bound for Trees) formula is used to balance exploration and exploitation:

[0092]

[0093] Among them, X j is the average return of node j, n is the number of visits to the parent node, n j is the number of visits to node j, C is the exploration parameter, and its initial value is set to 1.4. The C value can be adjusted dynamically according to the search process. For example, when the tree depth increases, the C value is gradually reduced to favor utilization.

[0094] Each time a simulation reaches a leaf node (after 14 days), an LSTM is used to predict the return of the final state. A discount factor γ = 0.95 is used for backtracking updates to balance short-term and long-term returns. After performing 1,000 simulations, the action sequence with the most visits is selected as the optimal long-term strategy.

[0095] Finally, daily treatment recommendations for the next two weeks are generated, including recommended stimulation parameters and expected effects. The system also takes into account the patient's schedule and optimizes the treatment time, such as avoiding the patient's work time or other important activities.

[0096] Step S13, data management and security. After each treatment, the session data is written to the local SQLite database 61. A data integrity check is performed to ensure that no data is lost or damaged. The local cache is updated, and the raw data of the last hour is saved in memory, and the minute-level summary data of the last 24 hours is saved in the disk cache to optimize the subsequent access speed.

[0097] At 2 a.m. every day (or when the device is idle), the system automatically synchronizes the newly added data to the cloud HDFS. Data is encrypted using the AES-256-GCM algorithm before transmission, and the key is managed by the HSM (Hardware Security Module). After the cloud receives the data, it immediately verifies the integrity and decrypts it for storage.

[0098] Spark jobs run data cleaning and transformation tasks daily, converting raw data into columnar storage formats (such as Parquet) and organizing them in a directory structure of / user / patient_id / year / month / day. The feature engineering pipeline runs once a week to prepare data for model training. A large-scale model retraining task is performed once a month to update global model parameters.

[0099] All data access is strictly managed through the RBAC (role-based access control) system. Doctors and researchers need to perform multi-factor authentication, including biometrics and dynamic passwords, when accessing. The system records all data access operations and monitors abnormal behavior in real time. If suspicious activity is detected (for example, a large amount of data is downloaded in a short period of time), the system will immediately lock the relevant account and notify the administrator.

[0100] Through the above steps, the adaptive electrical stimulation pelvic floor treatment system combined with reinforcement learning of the present invention can achieve precise personalized parameter control and long-term treatment optimization. The system can not only dynamically adjust the treatment parameters according to the patient's real-time feedback, but also continuously optimize the treatment strategy through long-term learning, significantly improving the treatment effect and patient experience of pelvic floor dysfunction. In addition, the data management and security measures of the present invention ensure patient privacy and data security, providing reliable technical support for large-scale clinical applications and research.

[0101] In order to verify the superiority of the adaptive electrical stimulation pelvic floor treatment system combined with reinforcement learning of the present invention, we designed a set of comparative experiments. The experiment includes one embodiment and three comparative examples, each representing a different treatment scheme. All experiments were conducted in the same clinical environment. The patients participating in the experiment were all women with moderate pelvic floor dysfunction, aged 45-60 years old, a total of 120, who were randomly divided into four groups, 30 people in each group. The experiment lasted for 12 weeks, with 3 treatments per week.

[0102] Example 1 adopts the adaptive electrical stimulation system of the present invention, including a reinforcement learning controller, multimodal biofeedback and a long-term optimization strategy generator.

[0103] Comparative Example 1 adopts traditional fixed parameter electrical stimulation treatment, and the parameter settings are: frequency 50 Hz, pulse width 300 μs, and the intensity is gradually increased to the patient's tolerance threshold.

[0104] Comparative Example 2 used simple feedback-regulated electrical stimulation therapy, and manually adjusted parameters based on the patient's subjective feedback.

[0105] Comparative Example 3 used electrical stimulation treatment based on a preset plan, and the closest preset plan was selected according to the patient's initial evaluation, and no adjustment was made during the treatment.

[0106] We selected the following five key indicators to evaluate treatment efficacy:

[0107] 1. Improvement of pelvic floor muscle strength: The maximum voluntary contraction force (MVC) was measured using the Myomed 932 biofeedback instrument, and the percentage of improvement before and after treatment was calculated.

[0108] 2. Improvement of urinary incontinence symptoms: The International Consultative Committee on Urinary Incontinence Questionnaire (ICIQ-UI SF) score was used to calculate the percentage of score reduction before and after treatment.

[0109] 3. Improvement in quality of life: The Pelvic Floor Dysfunction Impact Questionnaire (PFIQ-7) was used to evaluate the improvement in the score before and after treatment.

[0110] 4. Treatment compliance: record the percentage of patients completing the scheduled number of treatments.

[0111] 5. Incidence of adverse reactions: record the proportion of patients who experience discomfort, pain or other side effects during treatment.

[0112] After 12 weeks of treatment, we obtained the following results:

[0113]

[0114]

[0115] It can be clearly seen from the experimental results that Example 1 is significantly better than the three comparative examples in all indicators. We conduct an in-depth analysis of these results:

[0116] First, in terms of pelvic floor muscle strength improvement, Example 1 achieved a 68.5% improvement, which is much higher than other solutions. This shows that the adaptive electrical stimulation system of the present invention can more accurately locate and stimulate the target muscle group, and at the same time, by adjusting parameters in real time, the muscles can be stimulated in the best state, thereby maximizing the training effect.

[0117] Secondly, the significant improvement in urinary incontinence symptoms and quality of life (72.3% and 65.7%, respectively) reflects the excellent performance of this system in actual clinical applications. This is not only due to the improvement in muscle strength, but more importantly, the reinforcement learning algorithm can continuously optimize the treatment plan according to the unique response of each patient, achieving truly personalized treatment.

[0118] The improvement in treatment compliance (94.2%) is a particularly noteworthy result. This shows that the system of the present invention is not only more effective, but also significantly improves patients' enthusiasm for treatment. We believe that this is mainly due to the adaptive nature of the system and the design of the human-computer interface. Patients can see their progress in real time, and the treatment process is more comfortable, so they are naturally more willing to stick to the treatment.

[0119] Finally, the significant reduction in the incidence of adverse reactions (only 3.3%) highlights the safety advantage of this system, which is due to the multimodal biofeedback and real-time status assessment mechanism, which can timely detect and prevent potential discomfort or risks.

[0120] In summary, Example 1 is undoubtedly the best implementation scheme. It not only surpasses traditional methods in terms of treatment effect, but also makes significant progress in patient experience and safety. These results strongly prove the innovativeness and practical value of the adaptive electrical stimulation pelvic floor treatment system combined with reinforcement learning of the present invention. It opens up a new paradigm for the treatment of pelvic floor dysfunction, and is expected to significantly improve clinical treatment effects and improve the quality of life of patients.

[0121] It is worth mentioning that as the treatment progresses, the advantages of Example 1 continue to expand. This is due to the system's long-term optimization strategy generator, which can continuously adjust and improve the treatment plan based on historical data. We have reason to believe that in longer-term applications, the advantages of this system will be further highlighted.

[0122] The results of this study not only verify the superiority of the present invention, but also point out the direction for future development. For example, we can further optimize the reinforcement learning algorithm, explore more complex reward function designs, or introduce more biofeedback parameters to achieve more accurate personalized treatment. At the same time, the large amount of accumulated data also provides valuable resources for subsequent medical research, which helps us to have a deeper understanding of the mechanism and treatment principles of pelvic floor dysfunction.

[0123] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modification, replacement, and improvement made within the principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An adaptive electrical stimulation pelvic floor therapy system combined with reinforcement learning, characterized in that: include: A micro-electrical stimulation device for applying electrical stimulation to the pelvic floor muscles; a multimodal biofeedback sensor array for collecting physiological signals from the patient; A data processing module, connected to the multimodal biofeedback sensor array, for processing the collected physiological signals; A reinforcement learning controller connected to the data processing module and the micro-electrical stimulation device, and used to dynamically adjust the electrical stimulation parameters based on the processed physiological signals; a human-computer interaction interface connected to the reinforcement learning controller, and used to receive input from a doctor and display treatment information; The data storage management module is connected to the data processing module, the reinforcement learning controller and the human-computer interaction interface, and is used to store and manage treatment data.

2. The system according to claim 1, characterized in that The micro-electrical stimulation device includes: a multi-channel stimulation electrode array for contacting the pelvic floor muscles; a digital programmable waveform generator for generating an electrical stimulation waveform; a high-precision current source connected to the digital programmable waveform generator for converting the waveform into a current output; and an impedance detection circuit connected to the multi-channel stimulation electrode array for monitoring the electrode-tissue interface impedance.

3. The system according to claim 1, characterized in that The multimodal biofeedback sensor array includes: a surface electromyography sensor array for collecting electromyographic signals; a pressure sensor for detecting pelvic floor muscle pressure; an acceleration sensor for detecting patient posture; and a temperature sensor for monitoring local temperature changes.

4. The system according to claim 1, characterized in that The data processing module includes: a signal preprocessing unit for filtering and denoising the original signal; a feature extraction unit for extracting features from the preprocessed signal; and a data fusion unit for integrating multimodal data to generate a comprehensive state representation.

5. The system according to claim 1, characterized in that The reinforcement learning controller includes: a state evaluation module for evaluating the current pelvic floor muscle state; an Actor-Critic network for generating optimal stimulation parameters; and an experience playback buffer for storing historical experience data; wherein the Actor-Critic network is trained based on a reward function of multi-objective optimization.

6. The system according to claim 5, characterized in that The reinforcement learning controller also includes an adaptive noise injection mechanism for balancing exploration and exploitation.

7. The system according to claim 1, characterized in that It also includes a long-term optimization strategy generator, which includes: a multi-scale time series modeling module for predicting future state trends; and a Monte Carlo tree search module for generating long-term treatment strategies.

8. The system according to claim 1, characterized in that The human-computer interaction interface includes a doctor operation interface for setting treatment parameters and viewing analysis reports; a patient feedback interface for collecting patient subjective feelings; Data visualization module for displaying real-time treatment data and long-term rehabilitation progress.

9. The system according to claim 1, characterized in that The data storage management module includes: a local database for storing recent treatment data; a cloud data center for long-term data storage and large-scale analysis; and a data security module for ensuring the security of data transmission and storage.

10. A method of treating pelvic floor using the system according to any one of claims 1 to 9, characterized in that: The following steps are involved: Obtain the patient's basic information and initial status; Based on the basic information and the initial state, determining initial electrical stimulation parameters; The micro-electrical stimulation device is used to perform electrical stimulation therapy on a patient; the real-time physiological signals of the patient are collected by the multimodal biofeedback sensor array; the collected physiological signals are processed and feature extracted by the data processing module; based on the processed signals, the electrical stimulation parameters are dynamically adjusted by the reinforcement learning controller; the electrical stimulation, signal collection and parameter adjustment steps are repeated until a preset treatment time is reached; and a long-term treatment plan is generated based on historical treatment data by the long-term optimization strategy generator.

Images