Electrical stimulation signal generation method and system based on low-frequency electrotherapy and storage medium

Abstract

The invention discloses an electrical stimulation signal generation method and system based on low-frequency electrotherapy, and a storage medium, and relates to the technical field of electrical stimulation signal generation. The method comprises the following steps: generating an electrical stimulation signal through an electrical stimulation generation device, and carrying out the electrical stimulation of a part to be subjected to electrotherapy; acquiring an original electromyographic signal of the electrical stimulation part, preprocessing the original electromyographic signal to obtain a de-noised electromyographic signal, and calculating the muscle activation degree based on the de-noised electromyographic signal; establishing an intelligent stimulation model based on a Q-learning algorithm and the de-noised electromyographic signals, and outputting an electrical stimulation generation strategy based on the intelligent stimulation model; generating an electrical stimulation output parameter based on the electrical stimulation generation strategy, and outputting the electrical stimulation output parameter to electrical stimulation generation equipment to generate an electrical stimulation signal; the invention is used for solving the problem that the existing electrical stimulation signal generation technology cannot automatically generate and adjust the electrical stimulation signal by using the reinforcement learning technology while acquiring the electromyographic signal of the human body in real time and processing and analyzing the electromyographic signal.

Description

Technical Field

[0001] The present invention relates to the technical field of electrical stimulation signal generation, and specifically to a method, system and storage medium for generating electrical stimulation signals based on low-frequency electrotherapy. Background Art

[0002] Electrical stimulation signal generation technology refers to the technology of generating specific electrical pulses through current or voltage to generate electrical signals for stimulating biological tissues or the nervous system in order to treat diseases, improve functions or conduct scientific research. This technology is widely used in rehabilitation therapy, pain management, treatment of neurological diseases and scientific research.

[0003] Existing electrical stimulation signal generation technology usually has multi-channel output and can independently control the stimulation intensity and time of different electrodes. However, in applications of restoring muscle function, the muscles are usually stimulated step by step by manually selecting the frequency, amplitude and pulse width of the electrical stimulation signal. The frequency, amplitude and pulse width of the electrical stimulation signal are adjusted based on the pain perception of the electrical stimulation signal, which depends entirely on subjective perception. However, when the electrical stimulation signal is in a frequency and amplitude range with a therapeutic effect, the body's pain perception of the electrical stimulation signal is not obvious. It is easy to exceed the effective frequency and amplitude range by adjusting the electrical stimulation signal based on subjective perception, which cannot provide effective treatment and easily causes damage to the body's muscles. For example, a patent application with publication number CN116943032A discloses an electrical stimulation generation device, method, electronic device and storage medium. Although the solution is equipped with a central processing unit and a control module, it still requires manual selection of the electrical stimulation treatment plan, and the frequency and amplitude of the electrical stimulation signal cannot be dynamically adjusted during the electrical stimulation process. Therefore, the existing electrical stimulation signal generation technology cannot automatically generate and adjust the electrical stimulation signal using reinforcement learning technology while collecting the human body's electromyographic signals in real time and processing and analyzing them. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems in the prior art to a certain extent, by performing electrical stimulation through an electrical stimulation generating device; collecting original electromyographic signals at the electrical stimulation site, preprocessing the original electromyographic signals, and then calculating the muscle activation degree; establishing an intelligent stimulation model based on a Q-learning algorithm and denoised electromyographic signals, outputting an electrical stimulation generation strategy based on the intelligent stimulation model, and then using the electrical stimulation generating device to generate electrical stimulation signals; so as to solve the problem that the existing electrical stimulation signal generation technology is unable to collect the human body's electromyographic signals in real time, process and analyze them, and at the same time, use reinforcement learning technology to automatically generate and adjust electrical stimulation signals.

[0005] To achieve the above objectives, in a first aspect, the present application provides a method for generating an electrical stimulation signal based on low-frequency electrotherapy, comprising the following steps:

[0006] Generate an electrical stimulation signal by means of an electrical stimulation generating device to electrically stimulate the area to be treated with electrical stimulation;

[0007] Collecting the original electromyographic signal at the electrical stimulation site, preprocessing the original electromyographic signal to obtain a denoised electromyographic signal, and calculating the muscle activation degree based on the denoised electromyographic signal;

[0008] An intelligent stimulation model is established based on the Q-learning algorithm and denoised electromyographic signals, and an electrical stimulation generation strategy is output based on the intelligent stimulation model;

[0009] The electrical stimulation output parameters are generated based on the electrical stimulation generation strategy and are output to the electrical stimulation generation device to generate an electrical stimulation signal.

[0010] Furthermore, generating an electrical stimulation signal by an electrical stimulation generating device to electrically stimulate the electrotherapy site includes the following sub-steps:

[0011] The electrical stimulation signal generating device is used to initialize the electrical stimulation signal parameters, which include: signal frequency, signal amplitude, signal waveform, signal pulse width and electrotherapy time; an electrical stimulation signal is generated according to the initialized electrical stimulation signal parameters to electrically stimulate the electrotherapy site.

[0012] Furthermore, collecting the original electromyographic signal of the electrical stimulation site includes the following sub-steps:

[0013] Collect the electrical signal when the muscle has maximum autonomous contraction and mark it as maximum electromyographic signal; collect the electrical signal when the muscle is completely relaxed and mark it as minimum electromyographic signal; collect the electrical signal when the muscle is electrically stimulated and mark it as real-time electromyographic signal; mark the maximum electromyographic signal, minimum electromyographic signal and real-time electromyographic signal as original electromyographic signal.

[0014] Further, preprocessing the original electromyographic signal to obtain a denoised electromyographic signal includes the following sub-steps:

[0015] The original electromyographic signal is subjected to denoising processing by using the wavelet transform method to obtain a first processed signal; the first processed signal is filtered by using a Bastewal high-pass filter with a cutoff frequency of f1 to obtain a second processed signal, the second processed signal is full-wave rectified, and the full-wave rectified second processed signal is passed through a Bastewal low-pass filter with a cutoff frequency of f2 to obtain a denoised electromyographic signal, the denoised electromyographic signal including a preprocessed maximum electromyographic signal, a preprocessed minimum electromyographic signal and a preprocessed real-time electromyographic signal.

[0016] Furthermore, calculating the muscle activation degree based on the denoised electromyographic signal includes the following sub-steps:

[0017] The denoised EMG signal is normalized to obtain a normalized EMG signal. The neural activation intensity is calculated based on the normalized EMG signal to obtain the neural activation intensity u(t), where t represents the time t. The muscle activation degree is calculated based on the neural activation intensity u(t) to obtain the muscle activation degree a(t).

[0018] Furthermore, establishing an intelligent stimulation model based on the Q-learning algorithm and denoised electromyographic signals includes the following sub-steps:

[0019] The muscle activation degree of the current electrical stimulation site is defined as state Y, and the muscle activation degree is evenly divided into K parts according to the maximum muscle activation degree and the minimum muscle activation degree;

[0020] The step length of the signal frequency output by the electrical stimulation generating device is set to L1, the step length of the signal amplitude output by the electrical stimulation signal generating device is set to L2, the signal frequency and the signal amplitude are combined to obtain H types of output electrical stimulation signals, and the H types of output electrical stimulation signals are defined as action A;

[0021] Based on state Y and action A, establish the Q table of the Q-learning algorithm and construct the reward function R.

[0022] Furthermore, outputting an electrical stimulation generation strategy based on the intelligent stimulation model includes the following sub-steps:

[0023] Randomly initialize the Q table, obtain the current state of the electrical stimulation site, marked as y1, use the reward function R to calculate the reward value of the current state y1, marked as r1; then use the ε-greedy strategy to select the action corresponding to the maximum value corresponding to the state y1 in the Q table with a probability of 1-ε, or randomly select an action with a probability of ε, marked as a1; execute the selected action a1, obtain the next state y2 of the electrical stimulation site and the corresponding reward r2, calculate Q(y2, a2), Q(y2, a2) represents the corresponding value in the Q table when the state is y2 and the action is a2, and update Q(y2, a2) to the Q table; repeatedly iteratively train the Q-learning algorithm to obtain the optimal Q-learning algorithm, marked as the intelligent stimulation model, and output the electrical stimulation generation strategy in real time during the electrical stimulation process based on the intelligent stimulation model.

[0024] Furthermore, generating an electrical stimulation output parameter based on the electrical stimulation generation strategy and outputting the electrical stimulation output parameter to the electrical stimulation generation device to generate an electrical stimulation signal includes the following sub-steps:

[0025] The electrical stimulation signal generating device generates an initial electrical stimulation signal for electrical stimulation. The intelligent stimulation model obtains the state of the electrical stimulation site in real time, automatically selects the optimal action based on the state of the electrical stimulation site, outputs the electrical stimulation generation strategy, generates electrical stimulation output parameters based on the electrical stimulation generation strategy, and outputs them to the electrical stimulation generating device. The electrical stimulation signal generating device generates the corresponding electrical stimulation signal based on the electrical stimulation output parameters in real time for electrical stimulation therapy.

[0026] In a second aspect, the present application provides an electrical stimulation signal generation system based on low-frequency electrotherapy, including an electrical stimulation module, a signal acquisition module, an intelligent construction module, and an output management module;

[0027] The electrical stimulation module is used to generate an electrical stimulation signal to perform electrical stimulation treatment on the electrotherapy site;

[0028] The signal acquisition module includes a signal acquisition unit and a signal processing unit. The signal acquisition unit is used to collect the original electromyographic signal of the electrical stimulation site; the signal processing unit is used to pre-process the original electromyographic signal to obtain a denoised electromyographic signal, and calculate the muscle activation degree based on the denoised electromyographic signal;

[0029] The intelligent construction module includes a model construction unit and a strategy generation unit. The model construction unit establishes an intelligent stimulation model based on the Q-learning algorithm and denoised electromyographic signals; the strategy generation unit outputs an electrical stimulation generation strategy based on the intelligent stimulation model.

[0030] The output management module generates electrical stimulation output parameters based on the electrical stimulation generation strategy, and outputs the parameters to the electrical stimulation generation device to generate electrical stimulation signals.

[0031] In a third aspect, the present application provides a storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in the above method are performed.

[0032] Beneficial effects of the present invention: The present invention generates an electrical stimulation signal through an electrical stimulation generating device to perform electrical stimulation on a part to be treated with electrotherapy; then collects the original electromyographic signal of the part to be treated with electromyography, pre-processes the original electromyographic signal to obtain a denoised electromyographic signal, and calculates the muscle activation degree based on the denoised electromyographic signal; then establishes an intelligent stimulation model based on the Q-learning algorithm and the denoised electromyographic signal, and outputs an electrical stimulation generation strategy based on the intelligent stimulation model; finally, generates electrical stimulation output parameters based on the electrical stimulation generation strategy, and outputs them to the electrical stimulation generating device to generate an electrical stimulation signal; while collecting the electromyographic signal of the human body in real time and processing and analyzing it, the reinforcement learning technology can be used to automatically generate and adjust the electrical stimulation signal, so that the electrical stimulation always has a good therapeutic effect;

[0033] The present invention monitors electromyographic signals in real time, can understand the patient's immediate needs and reactions in real time, and adjust the frequency and intensity of electrical stimulation in time. At the same time, real-time monitoring can prevent muscle fatigue and damage caused by overstimulation; the wavelet transform method is used to denoise the electromyographic signals. Compared with traditional denoising methods, the wavelet transform method can better retain the morphology and characteristic information of the electromyographic signals while removing noise, thereby ensuring the accuracy of subsequent analysis; an intelligent stimulation model is established based on the Q-learning algorithm, and an electrical stimulation generation strategy is output based on the intelligent stimulation model. It will adapt to the different characteristics of different individual muscles to form the optimal strategy for the individual; as the rehabilitation process changes, the patient's physiological characteristics also change accordingly, and the change in physiological characteristics will inevitably lead to changes in the adaptive stimulation parameters. The characteristics of online training are to continuously optimize its own strategy based on this physiological change. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 is a functional block diagram of the system of the present invention;

[0035] Figure 2 is a flow chart of the steps of the method of the present invention;

[0036] Figure 3 It is the original electromyographic signal preprocessing flow chart of the present invention;

[0037] Figure 4 This is a graph showing the relationship between muscle activation and electrical stimulation intensity of the present invention;

[0038] Figure 5 It is the principle diagram of the Q-learning algorithm of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] Example 1, please refer to Figure 1 As shown, in the first aspect, the present application provides an electrical stimulation signal generation system based on low-frequency electrotherapy, including an electrical stimulation module, a signal acquisition module, an intelligent construction module and an output management module;

[0041] The electrical stimulation module is used to generate an electrical stimulation signal to perform electrical stimulation treatment on the electrotherapy site;

[0042] The electrical stimulation module is configured with an electrical stimulation strategy, which includes: initializing electrical stimulation signal parameters using an electrical stimulation signal generating device, the electrical stimulation signal parameters including: signal frequency, signal amplitude, signal waveform, signal pulse width and electrotherapy time;

[0043] In this embodiment, the signal frequency adjustment range is set to [20Hz, 50Hz]; because when the signal frequency is less than 10Hz, the muscles will hardly contract, and when the signal frequency is [15Hz, 60Hz], the number of contracted muscles gradually increases with the increase of frequency, and when the frequency is greater than 60Hz, the number of contracted muscles remains almost unchanged and maintains a stable state;

[0044] The signal amplitude directly affects the stimulation intensity. Whether the muscle can be excited and contracted and the magnitude of the contraction force are determined by the amplitude. In this embodiment, the signal amplitude setting range is [0, 20v]

[0045] Different electrical stimulation waveforms will show different effects on the human body; for example, the peak wave can promote blood circulation and improve tissue nutrition; the triangle wave can effectively increase the excitability of muscle tissue and has a good effect on the treatment of muscle atrophy; the square wave can make the energy output more stable and safe, and has outstanding effects in promoting tissue absorption and restoring muscle function. It is often used for analgesia, neuromuscular electrical stimulation and regulating autonomic nerve function. In this embodiment, the electrical stimulation signal is a square wave signal;

[0046] The signal pulse width refers to the duration of the pulse electrical signal, which directly affects the stimulation intensity. The larger the width, the greater the stimulation to the patient. To cause excitement in human tissue, the stimulation intensity needs to reach a certain threshold, but at the same time, in order to avoid causing adverse reactions to the human body, the stimulation intensity cannot be too large. In this embodiment, the pulse width adjustment range is set to [200us, 1800us];

[0047] Generate an electrical stimulation signal according to the initialized electrical stimulation signal parameters to electrically stimulate the area to be treated with electrical stimulation;

[0048] In the specific implementation process, the low-frequency pulse current can excite the nerve and muscle tissue, promote local blood circulation, and has the characteristic of not obvious thermal effect on human tissue. When the motor nerves or muscles are stimulated by low-frequency pulse current, it can cause muscle contraction, and has the effect of accelerating nerve recovery and regeneration. Through electrical stimulation, the contraction and relaxation state of the muscles can be changed, and the muscles can be induced to simulate normal muscles for autonomous movement, so as to improve or restore the function of the stimulated muscles; for safety reasons, the initial electrical stimulation signal can be set to a weaker electrical stimulation signal.

[0049] The signal acquisition module includes a signal acquisition unit and a signal processing unit. The signal acquisition unit is used to collect the original electromyographic signal of the electrical stimulation site; the signal processing unit is used to pre-process the original electromyographic signal to obtain a denoised electromyographic signal, and calculate the muscle activation degree based on the denoised electromyographic signal;

[0050] The signal acquisition unit is configured with a signal acquisition strategy, which includes: ensuring that the skin surface of the acquisition site is clean, placing the electrode above the belly of the muscle at the acquisition site, along the direction of the muscle fiber, ensuring good contact between the electrode and the skin, connecting the electrode to the electromyographic acquisition device, collecting the electrical signal when the muscle has maximum autonomous contraction, and marking it as the maximum electromyographic signal; collecting the electrical signal when the muscle is completely relaxed, and marking it as the minimum electromyographic signal; collecting the electrical signal when the muscle is electrically stimulated, and marking it as the real-time electromyographic signal; marking the maximum electromyographic signal, the minimum electromyographic signal and the real-time electromyographic signal as the original electromyographic signal; for example, to collect the maximum electromyographic signal at the ulnar extensor carpi of the right arm, place the electrode above the belly of the ulnar extensor carpi, along the direction of the muscle fiber, ensuring good contact between the electrode and the skin, connecting the electrode to the electromyographic acquisition device, and then forcefully extend the wrist to collect the electrical signal of the muscle, and marking it as the maximum electromyographic signal; when collecting the minimum electromyographic signal at the ulnar extensor carpi of the right arm, let the arm hang naturally and relax completely;

[0051] In the specific implementation process, there are two main ways to collect electromyographic signals: probe type and electrode sheet type. Most of them are to obtain the electrical signals generated by the muscles on the body surface by sticking a small electromyographic signal collection module, namely, electrode sheet type, on the muscle surface. Compared with the probe type electromyographic signal collection method, although the electrode sheet type has a slight difference in signal accuracy, it is far superior to the probe type electromyographic signal collection in terms of non-invasiveness, repeatability, portability, economy and acceptability. In this embodiment, the electrode sheet type is selected to collect the surface electromyographic signals of the relevant muscles;

[0052] The signal processing unit is configured with a signal processing strategy, which includes: Figure 3As shown, the original electromyographic signal is preprocessed: the original electromyographic signal is subjected to denoising processing by using the wavelet transform method to obtain a first processed signal; the principle formula for denoising the original electromyographic signal by using the wavelet transform method is: s(n)=f(n)+e(n), wherein s(n) is the original electromyographic signal, e(n) is Gaussian white noise, and f(n) is the original electromyographic signal after denoising, that is, the noise part in the original electromyographic signal s(n) is filtered out by filtering out the Gaussian white noise e(n), and the denoised original electromyographic signal f(n) is obtained. In this embodiment, the db4 wavelet is selected to perform a 5-layer decomposition on the original electromyographic signal, and a suitable threshold is set for the obtained wavelet coefficients, and all wavelet coefficients are set to zero and classified below the relevant threshold; finally, the signal is reconstructed by using the inverse wavelet transform based on the modified wavelet coefficients to obtain the denoised original electromyographic signal, which is marked as the denoised electromyographic signal;

[0053] The first processed signal is filtered using a Bastevo high-pass filter with a cutoff frequency of f1 to remove motion artifacts. Artifacts in the electromyographic signal refer to other electrical activities recorded in the electromyographic acquisition in addition to the electrical activities of skeletal muscles. For example, during electrical stimulation, the stimulation current diffuses through the tissues, organs or electrolyte solutions inside and outside the body to the recording electrode and is guided, amplified and recorded, forming stimulation artifacts. The second processed signal is obtained after filtering. In this embodiment, the cutoff frequency f1 is 30 Hz;

[0054] The second processed signal is full-wave rectified, that is, the absolute value of the second processed signal is taken, the negative half axis of the second processed signal is flipped to the positive half axis, and the second processed signal after full-wave rectification is passed through a Bastewo low-pass filter with a cutoff frequency of f2 to obtain a denoised electromyographic signal. In this embodiment, the cutoff frequency f2 is 5 Hz, which can simulate the characteristics of the muscle low-pass filter, improve the quality and reliability of signal processing, and thus provide accurate data support for subsequent processing and analysis;

[0055] The denoised electromyographic signal includes a preprocessed maximum electromyographic signal, a preprocessed minimum electromyographic signal and a preprocessed real-time electromyographic signal;

[0056] Normalizing the denoised electromyographic signal, preprocessing the maximum electromyographic signal during the maximum voluntary contraction to obtain the preprocessed maximum electromyographic signal, and then finding the maximum value of the maximum voluntary contraction signal, and regarding the maximum value as the electromyographic signal when the muscle activation degree is 100%; dividing the denoised electromyographic signal, including the preprocessed maximum electromyographic signal, the preprocessed minimum electromyographic signal, and the preprocessed real-time electromyographic signal, by the maximum value to obtain a normalized electromyographic signal, wherein the normalized electromyographic signal includes a normalized maximum electromyographic signal, a normalized minimum electromyographic signal, and a normalized real-time electromyographic signal;

[0057] The neural activation strength is calculated based on the normalized electromyographic signal to obtain the neural activation strength u(t), where t represents time t. The formula for calculating the neural activation strength is: u(t)=εa(td)-λ1u(t-1)-λ2u(t-2), wherein u(t) represents the neural activation strength at the electrical stimulation site at time t, a(td) represents the normalized real-time electromyographic signal at the electrical stimulation site at time t, t represents the acquisition time of the normalized real-time electromyographic signal, d represents the delay time of the electrode, ε represents the gain coefficient, ε=λ1+λ2+1.0, λ1 and λ2 represent recursive coefficients, and in this embodiment, d=10ms, λ1=0.5, and λ2=0.5;

[0058] The muscle activation degree is calculated based on the nerve activation intensity u(t) to obtain the muscle activation degree a(t); the formula for calculating the muscle activation degree is: Where a(t) represents the muscle activation at time t, e is a mathematical constant, A is a nonlinear shape coefficient, representing the nonlinearity of the neural activation intensity u(t) and the muscle activation intensity a(t), and the value range is [-3, 0]. When A=0, it means that the neural activation intensity u(t) and the muscle activation intensity a(t) are linearly related. In this embodiment, A=-1.5;

[0059] In the specific implementation process, the original electromyographic signal collected by the electromyographic signal acquisition device contains more noise interference. Before performing electromyographic signal feature analysis, the electromyographic signal must first be denoised. According to the different types of noise, bandpass filtering, wavelet decomposition, empirical mode decomposition and other methods can be used to remove noise, and the appropriate denoising method can be flexibly selected according to actual needs; the wavelet transform used in this embodiment simultaneously analyzes the signal in the time and frequency domains, can accurately remove noise and retain the original signal, and is very suitable for denoising non-stationary nonlinear signals due to its multi-resolution, decorrelation and flexibility. When preprocessing the original electromyographic signal, the maximum electromyographic signal, the minimum electromyographic signal and the real-time electromyographic signal must be preprocessed separately. The maximum electromyographic signal and the minimum electromyographic signal do not need to be collected in real time at the same muscle site, and are collected according to the usage. For example, when using it, 3 days is a treatment cycle, then the maximum electromyographic signal and the minimum electromyographic signal are collected once every three days, and preprocessed.

[0060] The intelligent construction module includes a model construction unit and a strategy generation unit. The model construction unit establishes an intelligent stimulation model based on the Q-learning algorithm and denoised electromyographic signals; the strategy generation unit outputs an electrical stimulation generation strategy based on the intelligent stimulation model.

[0061] The model building unit is configured with a model building strategy, which includes: defining the muscle activation degree of the current electrical stimulation site as state Y, and dividing the muscle activation degree into K parts on average according to the maximum muscle activation degree and the minimum muscle activation degree; in this embodiment, the muscle activation degree is divided into 40 parts on average,

[0062] The step size of the signal frequency output by the electrical stimulation generating device is set to L1, the step size of the signal amplitude output by the electrical stimulation signal generating device is set to L2, the signal frequency and the signal amplitude are combined to obtain H kinds of outputtable electrical stimulation signals, and the H kinds of outputtable electrical stimulation signals are defined as action A; for example, in this embodiment, the signal frequency adjustment range is set to [20Hz, 50Hz], the step size L1=2Hz, the signal amplitude adjustment range is set to [0, 20v], and the step size of the signal amplitude is set to L2=1v, so H=15*20=300, that is, 300 kinds of outputtable electrical stimulation signals;

[0063] In the specific implementation process, the size of the electromyographic signal generated by the electrical stimulation causing the muscle contraction, that is, the degree of muscle activation, is affected by the signal frequency and signal amplitude of the electrical stimulation signal. The signal frequency and signal amplitude determine the intensity of the electrical stimulation. Please refer to Figure 4 As shown in the figure, when the electrical stimulation is less than the critical value, the muscle activation remains unchanged, that is, the muscle cannot generate enough force. The area below the critical value is called the unactivated area. When the electrical stimulation intensity exceeds the critical value of the unactivated area, the muscle activation increases with the increase of the electrical stimulation intensity. This part of the area is called the activated area. When the stimulation intensity is too large, the muscle activation degree remains unchanged with the increase of the electrical stimulation intensity. This part is the saturation area. The relationship between the muscle activation and the electrical stimulation intensity in different areas is different. When performing electrical stimulation, it is necessary to select appropriate stimulation intensity from different areas according to needs.

[0064] A Q-learning algorithm Q table is established based on state Y and action A, and a reward function R is constructed based on muscle activation and human perception; in this embodiment, the reward function is: R = p*(ηΔb+μs), where p represents human pain, and its value is -1 and 1, 1 represents no pain, and -1 represents pain, η and μ are weight factors, and their value range is [0, 1], Δb represents the difference between the current output electrical stimulation and the next state output electrical stimulation, and s represents the degree of discomfort during the electrical stimulation, and its value range is [-1, 0];

[0065] Randomly initialize the Q table, obtain the state of the current electrical stimulation site, marked as y1, use the reward function R to calculate the reward value of the current state y1, marked as r1; then use the ε-greedy strategy to select the action corresponding to the maximum value corresponding to the state y1 in the Q table with a probability of 1-ε, or randomly select an action with a probability of ε, marked as A1; execute the selected action A1, in this embodiment ε = 0.2, for example, some data of the randomly initialized Q table are shown in Table 1 below,

[0066] Table 1: Partial data of Q table

[0067] Q a1 a2 a3 a4 y1 2 -2 0 3 y2 4 1 3 1 y3 1 4 3 0

[0068] The state of the electrically stimulated site is y1. Find the row where state y1 is located and find that Q(y1, a4)>Q(y1, a1)>Q(y1, a3)>Q(y1, a2). Therefore, action a4 is selected with a probability of 1-ε. An action is randomly selected from a1, a2, a3, and a4 with a probability of ε.

[0069] Get the next state y2 and the corresponding reward r2 of the electrical stimulation site, calculate Q(y2, A2), Q(y2, A2) represents the corresponding value in the Q table when the state is y2 and the action is A2, and update Q(y2, A2) to the Q table; repeatedly iterate the Q-learning algorithm to obtain the optimal Q-learning algorithm, which is marked as the intelligent stimulation model;

[0070] The strategy generation unit is configured with a strategy generation strategy, which includes: obtaining real-time electromyographic signals of the electrical stimulation site during the electrical stimulation process, performing preprocessing to obtain muscle activation, and inputting the signals into the intelligent stimulation model, and the intelligent stimulation model outputting the electrical stimulation generation strategy in real time;

[0071] In the specific implementation process, please refer to Figure 5As shown, the core of the basic principle of the Q-learning algorithm is to learn an action value function. In this embodiment, the action value function Q is: Q(Y,A)=R(Y,A)-γ*max{Q(Y',A')}, where Q(Y,A) represents the state action value value when the state is s and the action is a, indicating the expected utility or value of performing action a in state s; Y and A are the current state and action of the electrical stimulation; Y' and A' represent the state and action of the next electrical stimulation; Q(Y',A') represents the expected utility or value of performing action A' in state Y'; R(Y,A) represents the reward for performing action a in state s. ; γ is a discount factor, which determines the impact of future rewards on the current Q value; when γ is close to 1, it indicates that future rewards are of high importance and the algorithm is more inclined to long-term returns; and when γ is close to 0, it indicates that the algorithm only considers short-term rewards. Usually γ is set between 0 and 1. In this embodiment, γ is 0.6; the function represents the maximum expected benefit that can be obtained by taking action a in state y; this function is usually represented by a table, namely the Q table. In this embodiment, the rows in the Q table represent different states, the columns represent different actions, and the value in each cell is obtained through continuous iterative updates. Its goal is to find a strategy that can obtain the maximum long-term return.

[0072] The output management module generates electrical stimulation output parameters based on the electrical stimulation generation strategy, and outputs the parameters to the electrical stimulation generation device to generate electrical stimulation signals;

[0073] The output management module is configured with an output management strategy, which includes: the electrical stimulation signal generating device generates an initial electrical stimulation signal for electrical stimulation, the intelligent stimulation model obtains the state of the electrical stimulation site in real time, automatically selects the optimal action based on the state of the electrical stimulation site, outputs the electrical stimulation generation strategy, generates electrical stimulation output parameters based on the electrical stimulation generation strategy, and outputs them to the electrical stimulation generating device, and the electrical stimulation signal generating device generates a corresponding electrical stimulation signal in real time based on the electrical stimulation output parameters for electrical stimulation treatment;

[0074] In the specific implementation process, a computer device can be used to write a program to automatically analyze the electrical stimulation generation strategy, obtain electrical stimulation output parameters, generate corresponding electrical stimulation output instructions based on the electrical stimulation output parameters, send them to the electrical stimulation signal generation device, and output the corresponding electrical stimulation signal.

[0075] Example 2, please refer to Figure 2 As shown, the present application provides a method for generating an electrical stimulation signal based on low-frequency electrotherapy, comprising the following steps:

[0076] Step S1, generating an electrical stimulation signal by an electrical stimulation generating device to electrically stimulate the part to be treated with electrical stimulation; Step S1 includes the following sub-steps:

[0077] Step S101, using an electrical stimulation signal generating device to initialize electrical stimulation signal parameters, the electrical stimulation signal parameters including: signal frequency, signal amplitude, signal waveform, signal pulse width and electrotherapy time;

[0078] Step S102, generating an electrical stimulation signal according to the initialized electrical stimulation signal parameters to electrically stimulate the area to be treated with electrical stimulation.

[0079] Step S2, collecting the original electromyographic signal of the electrical stimulation site, preprocessing the original electromyographic signal to obtain a denoised electromyographic signal, and calculating the muscle activation degree based on the denoised electromyographic signal; Step S2 includes the following sub-steps:

[0080] Step S201, collecting the original electromyographic signal of the electrical stimulation site, step S201 includes the following sub-steps:

[0081] Ensure that the skin surface of the collection site is clean, and place the electrode on the belly of the muscle at the collection site, along the direction of the muscle fiber, to ensure good contact between the electrode and the skin;

[0082] Step S2011, connecting the electrodes to an electromyographic acquisition device, collecting the electrical signal of the muscle at the maximum spontaneous contraction, and marking it as the maximum electromyographic signal;

[0083] Step S2012, collecting the electrical signal when the muscle is completely relaxed, and marking it as the minimum electromyographic signal; collecting the electrical signal when the muscle is electrically stimulated, and marking it as the real-time electromyographic signal;

[0084] Step S2013, marking the maximum electromyographic signal, the minimum electromyographic signal and the real-time electromyographic signal as original electromyographic signals;

[0085] Step S202, preprocessing the original electromyographic signal to obtain a denoised electromyographic signal, and step S202 includes the following sub-steps:

[0086] Step S2021, using wavelet transform method to perform noise reduction processing on the original electromyographic signal to obtain a first processed signal;

[0087] Step S2022, using a Bastewell high-pass filter with a cutoff frequency of f1 to filter the first processed signal to obtain a second processed signal;

[0088] Step S2023, full-wave rectifying the second processed signal;

[0089] Step S2024, the second processed signal after full-wave rectification is passed through a Bastewell low-pass filter with a cutoff frequency of f2 to obtain a denoised electromyographic signal, wherein the denoised electromyographic signal includes a preprocessed maximum electromyographic signal, a preprocessed minimum electromyographic signal and a preprocessed real-time electromyographic signal;

[0090] Step S203, normalizing the denoised electromyographic signal to obtain a normalized electromyographic signal;

[0091] Step S204, calculating the neural activation strength based on the normalized electromyographic signal to obtain the neural activation strength u(t), where t represents time t;

[0092] Step S205, calculating the muscle activation degree based on the nerve activation intensity u(t) to obtain the muscle activation degree a(t).

[0093] Step S3, establishing an intelligent stimulation model based on the Q-learning algorithm and the denoised electromyographic signal, and outputting an electrical stimulation generation strategy based on the intelligent stimulation model; Step S3 includes the following sub-steps:

[0094] Step S301, defining the muscle activation degree of the current electrical stimulation site as state Y, and dividing the muscle activation degree into K parts according to the maximum muscle activation degree and the minimum muscle activation degree;

[0095] Step S302, setting the step length of the signal frequency output by the electrical stimulation signal generating device to L1, and setting the step length of the signal amplitude output by the electrical stimulation signal generating device to L2;

[0096] Step S303, combining the signal frequency and the signal amplitude to obtain H types of outputtable electrical stimulation signals, and defining the H types of outputtable electrical stimulation signals as action A;

[0097] Step S304, establishing a Q table of the Q-learning algorithm based on the state Y and the action A, and constructing a reward function R based on the muscle activation degree and the human body feeling;

[0098] Step S305, outputting an electrical stimulation generation strategy based on the intelligent stimulation model; Step S305 includes the following sub-steps:

[0099] Step S3051, randomly initialize the Q table to obtain the state of the current electrical stimulation site, marked as y1;

[0100] Step S3052, using the reward function R to calculate the reward value obtained by the current state y1, marked as r1;

[0101] Step S3053, using the ε-greedy strategy to select the action corresponding to the maximum value corresponding to state y1 in the Q table with a probability of 1-ε, or randomly select an action with a probability of ε, marked as a1;

[0102] Step S3054, execute the selected action a1, obtain the next state y2 of the electrical stimulation site and the corresponding reward r2, calculate Q(y2, a2), Q(y2, a2) represents the corresponding value in the Q table when the state is y2 and the action is a2;

[0103] Step S3055, update Q(y2, a2) into the Q table;

[0104] Step S3056, repeatedly iteratively train the Q-learning algorithm to obtain the optimal Q-learning algorithm, which is marked as the intelligent stimulation model.

[0105] Step S3057, outputting the electrical stimulation generation strategy in real time during the electrical stimulation process based on the intelligent stimulation model.

[0106] Step S4, generating electrical stimulation output parameters based on the electrical stimulation generation strategy, and outputting the parameters to the electrical stimulation generation device to generate electrical stimulation signals; Step S4 includes the following sub-steps:

[0107] Step S401, the electrical stimulation signal generating device generates an initial electrical stimulation signal for electrical stimulation, and the intelligent stimulation model obtains the state of the electrical stimulation site in real time;

[0108] Step S402, automatically selecting the optimal action based on the state of the electrical stimulation site and outputting the electrical stimulation generation strategy;

[0109] Step S403, generating electrical stimulation output parameters based on the electrical stimulation generation strategy, and outputting the parameters to the electrical stimulation generation device;

[0110] Step S404: the electrical stimulation signal generating device generates a corresponding electrical stimulation signal in real time based on the electrical stimulation output parameter to perform electrical stimulation therapy.

[0111] Embodiment 3. The present application also provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the method for generating an electrical stimulation signal based on low-frequency electrotherapy provided by the above-mentioned methods, and the method includes: generating an electrical stimulation signal by an electrical stimulation generating device to electrically stimulate the electrotherapy site; collecting the original electromyographic signal of the electrostimulation site, preprocessing the original electromyographic signal to obtain a denoised electromyographic signal, and calculating the muscle activation degree based on the denoised electromyographic signal; establishing an intelligent stimulation model based on the Q-learning algorithm and the denoised electromyographic signal, and outputting an electrical stimulation generation strategy based on the intelligent stimulation model; generating electrical stimulation output parameters based on the electrical stimulation generation strategy, and outputting them to the electrical stimulation generating device to generate an electrical stimulation signal.

[0112] Embodiment 4. The present application also provides a computer-readable storage medium. The present application provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the method for generating an electrical stimulation signal based on low-frequency electrotherapy are executed to achieve the following functions: an electrical stimulation generating device generates an electrical stimulation signal to electrically stimulate the electrotherapy site; the original electromyographic signal of the electrical stimulation site is collected, the original electromyographic signal is preprocessed to obtain a denoised electromyographic signal, and the muscle activation degree is calculated based on the denoised electromyographic signal; an intelligent stimulation model is established based on the Q-learning algorithm and the denoised electromyographic signal, and an electrical stimulation generation strategy is output based on the intelligent stimulation model; electrical stimulation output parameters are generated based on the electrical stimulation generation strategy, and output to the electrical stimulation generating device to generate an electrical stimulation signal.

[0113] Through the description of the above implementation methods, the embodiments of the present invention can be provided as methods, systems or computer program products. Based on such an understanding, the above technical solutions can be essentially or partly contributed to the prior art in the form of software products, which can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and include several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0114] In the embodiments provided in the present application, it should be understood that the disclosed system or method can be implemented in other ways. The embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of systems, modules and units can be electrical, mechanical or other forms.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for generating an electrical stimulation signal based on low-frequency electrotherapy, characterized in that: The steps include: Generate an electrical stimulation signal by means of an electrical stimulation generating device to electrically stimulate the area to be treated with electrical stimulation; Collecting the original electromyographic signal at the electrical stimulation site, preprocessing the original electromyographic signal to obtain a denoised electromyographic signal, and calculating the muscle activation degree based on the denoised electromyographic signal; An intelligent stimulation model is established based on the Q-learning algorithm and denoised electromyographic signals, and an electrical stimulation generation strategy is output based on the intelligent stimulation model; The electrical stimulation output parameters are generated based on the electrical stimulation generation strategy and are output to the electrical stimulation generation device to generate an electrical stimulation signal.

2. The method for generating an electrical stimulation signal based on low-frequency electrotherapy according to claim 1, characterized in that: Generating an electrical stimulation signal by an electrical stimulation generating device to electrically stimulate the electrotherapy site includes the following sub-steps: The electrical stimulation signal generating device is used to initialize the electrical stimulation signal parameters, which include: signal frequency, signal amplitude, signal waveform, signal pulse width and electrotherapy time; an electrical stimulation signal is generated according to the initialized electrical stimulation signal parameters to electrically stimulate the electrotherapy site.

3. The method for generating an electrical stimulation signal based on low-frequency electrotherapy according to claim 2, characterized in that: Collecting the original electromyographic signal of the electrical stimulation site includes the following sub-steps: Collect the electrical signal when the muscle has the maximum spontaneous contraction, marked as the maximum electromyographic signal; collect the electrical signal when the muscle is completely relaxed, marked as the minimum electromyographic signal; Collect the electrical signals when the muscles are electrically stimulated and mark them as real-time electromyographic signals; The maximum EMG signal, the minimum EMG signal and the real-time EMG signal are marked as the original EMG signals.

4. The method for generating an electrical stimulation signal based on low-frequency electrotherapy according to claim 3, characterized in that: Preprocessing the original electromyographic signal to obtain a denoised electromyographic signal includes the following sub-steps: The original electromyographic signal is subjected to denoising processing by using the wavelet transform method to obtain a first processed signal; the first processed signal is filtered by using a Bastewal high-pass filter with a cutoff frequency of f1 to obtain a second processed signal, the second processed signal is full-wave rectified, and the full-wave rectified second processed signal is passed through a Bastewal low-pass filter with a cutoff frequency of f2 to obtain a denoised electromyographic signal, the denoised electromyographic signal including a preprocessed maximum electromyographic signal, a preprocessed minimum electromyographic signal and a preprocessed real-time electromyographic signal.

5. The method for generating an electrical stimulation signal based on low-frequency electrotherapy according to claim 4, characterized in that: Calculating muscle activation based on denoised electromyographic signals includes the following sub-steps: The denoised EMG signal is normalized to obtain a normalized EMG signal. The neural activation intensity is calculated based on the normalized EMG signal to obtain the neural activation intensity u(t), where t represents the time t. The muscle activation degree is calculated based on the neural activation intensity u(t) to obtain the muscle activation degree a(t).

6. The method for generating an electrical stimulation signal based on low-frequency electrotherapy according to claim 5, characterized in that: The establishment of an intelligent stimulation model based on the Q-learning algorithm and denoised electromyographic signals includes the following sub-steps: The muscle activation degree of the current electrical stimulation site is defined as state Y, and the muscle activation degree is evenly divided into K parts according to the maximum muscle activation degree and the minimum muscle activation degree; The step length of the signal frequency output by the electrical stimulation generating device is set to L1, the step length of the signal amplitude output by the electrical stimulation signal generating device is set to L2, the signal frequency and the signal amplitude are combined to obtain H types of output electrical stimulation signals, and the H types of output electrical stimulation signals are defined as action A; Based on state Y and action A, establish the Q table of the Q-learning algorithm and construct the reward function R.

7. The method for generating an electrical stimulation signal based on low-frequency electrotherapy according to claim 6, characterized in that: The strategy for generating electrical stimulation based on the intelligent stimulation model output includes the following sub-steps: Randomly initialize the Q table, obtain the state of the current electrical stimulation site, marked as y1, use the reward function R to calculate the reward value of the current state y1, marked as r1; then use the ε-greedy strategy to select the action corresponding to the maximum value corresponding to the state y1 in the Q table with a probability of 1-ε, or randomly select an action with a probability of ε, marked as a1; execute the selected action a1, obtain the next state y2 of the electrical stimulation site and the corresponding reward r2, calculate Q(y2, a2), Q(y2, a2) represents the corresponding value in the Q table when the state is y2 and the action is a2, and update Q(y2, a2) to the Q table; The Q-learning algorithm is iteratively trained repeatedly to obtain an optimal Q-learning algorithm, which is marked as an intelligent stimulation model. Based on the intelligent stimulation model, an electrical stimulation generation strategy is output in real time during the electrical stimulation process.

8. The method for generating an electrical stimulation signal based on low-frequency electrotherapy according to claim 7, characterized in that: Generating electrical stimulation output parameters based on the electrical stimulation generation strategy and outputting them to the electrical stimulation generation device to generate electrical stimulation signals includes the following sub-steps: The electrical stimulation signal generating device generates an initial electrical stimulation signal for electrical stimulation. The intelligent stimulation model obtains the state of the electrical stimulation site in real time, automatically selects the optimal action based on the state of the electrical stimulation site, outputs the electrical stimulation generation strategy, generates electrical stimulation output parameters based on the electrical stimulation generation strategy, and outputs them to the electrical stimulation generating device. The electrical stimulation signal generating device generates the corresponding electrical stimulation signal based on the electrical stimulation output parameters in real time for electrical stimulation therapy.

9. A system for generating an electrical stimulation signal based on low-frequency electrotherapy, used to implement the method for generating an electrical stimulation signal based on low-frequency electrotherapy according to any one of claims 1 to 8, characterized in that: It includes an electrical stimulation module, a signal acquisition module, an intelligent construction module and an output management module; The electrical stimulation module is used to generate an electrical stimulation signal to perform electrical stimulation treatment on the part to be treated; The signal acquisition module includes a signal acquisition unit and a signal processing unit, the signal acquisition unit is used to collect the original electromyographic signal of the electrical stimulation site; the signal processing unit is used to pre-process the original electromyographic signal to obtain a denoised electromyographic signal, and calculate the muscle activation degree based on the denoised electromyographic signal; The intelligent construction module includes a model construction unit and a strategy generation unit. The model construction unit establishes an intelligent stimulation model based on the Q-learning algorithm and the denoised electromyographic signal; the strategy generation unit outputs an electrical stimulation generation strategy based on the intelligent stimulation model; The output management module generates electrical stimulation output parameters based on the electrical stimulation generation strategy, and outputs the parameters to the electrical stimulation generation device to generate electrical stimulation signals.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are executed.

Images