Multi-source information fusion intelligent rehabilitation glove system

Abstract

The invention provides a multi-source information fusion intelligent rehabilitation glove system. A main control unit of the system is used for coordinating and controlling all parts of rehabilitation gloves; the charging interface charges the battery; the control panel provides an interactive interface between a user and the system; the servo linear motor is used for controlling movement and position feedback of the rehabilitation glove; the battery supplies power to the system; the LED lamp is used for visual stimulation; the buzzer is used for audio stimulation; the low-frequency electrical stimulation interface is used for electrical stimulation; the multi-source fusion control module is used for monitoring a pressure signal and a muscle activity potential signal of the glove and controlling the movement and strength of the glove in real time; and wireless communication is used for data transmission and remote monitoring. By integrating multi-source information sensory stimulation, the rehabilitation process is enhanced, and particularly, the rehabilitation of damaged nerves is facilitated. Secondly, the multi-source information fusion control module monitors and adjusts hand movement and muscle movement of the patient in real time through a pressure sensor and a myoelectricity sensor, and personalized rehabilitation support is provided.

Description

Technical Field

[0001] The present invention relates to the field of rehabilitation and medical technology, and in particular to a multi-source information fusion intelligent rehabilitation glove system. Background Art

[0002] Existing rehabilitation equipment usually focuses on single sensory stimulation, such as electromyography, and ignores the synergistic effect of multiple senses. This leads to limited rehabilitation effects on patients' damaged nerves, because multi-source information sensory stimulation (including auditory, visual, tactile electrical stimulation) plays an important role in promoting nerve regeneration and rehabilitation, lacks coordinated control of multi-source information, and cannot adapt to individual differences and rehabilitation progress of patients.

[0003] Existing rehabilitation products and equipment lack sufficient customization and personalization options to meet the specific needs of different patients, which limits the efficiency and effectiveness of the rehabilitation process.

[0004] Existing rehabilitation equipment has problems with safety and comfort, which can cause patients to feel uncomfortable when using it and even resist the rehabilitation process. Multi-source information sensory stimulation and fusion control can improve the user-friendliness of rehabilitation equipment.

[0005] In summary, existing rehabilitation technologies and products have defects and shortcomings in multi-source information sensory stimulation and multi-source information fusion control. Summary of the invention

[0006] In order to achieve the above-mentioned purpose and other advantages of the present invention, the purpose of the present invention is to provide a multi-source information fusion intelligent rehabilitation glove system, including a main control unit, a charging interface, a control panel, a plurality of servo linear motors, a battery, an LED light, a buzzer, a multi-source fusion control module, a wireless communication module, and a low-frequency electrical stimulation interface; wherein,

[0007] The main control unit is used to coordinate and control various parts of the rehabilitation glove;

[0008] The charging interface is used to charge the battery;

[0009] The control panel is used to provide an interactive interface between the user and the system;

[0010] The servo linear motor is used to control the movement and position feedback of the rehabilitation glove;

[0011] The battery is used to provide power to the system;

[0012] The LED light is installed at the distal end of the fingers in the back of the hand direction of the rehabilitation glove for visual stimulation;

[0013] The buzzer is used for audio stimulation;

[0014] The low-frequency electrical stimulation interface is used for electrical stimulation rehabilitation training;

[0015] The multi-source fusion control module is used to monitor the pressure signal of the glove and the muscle action potential signal, and to control the movement and strength of the glove in real time according to the monitored pressure sensing signal and muscle action potential signal;

[0016] The wireless communication is used for data transmission and remote monitoring.

[0017] Furthermore, the control panel is used to adjust and configure the stimulation mode and frequency.

[0018] Furthermore, the multi-source fusion control module includes a pressure sensor and an electromyographic sensor, wherein the pressure sensor is used to monitor the pressure signal of the glove, and the electromyographic sensor is used to monitor the muscle activity potential signal.

[0019] Furthermore, the pressure sensor is a resistive thin film pressure sensor.

[0020] Furthermore, the multi-source fusion control module controls the movement and strength of the glove in real time according to the monitored pressure sensing signal and muscle action potential signal, including the following steps:

[0021] Pre-process the monitored pressure signals and muscle action potential signals;

[0022] Extract features from pressure signals and muscle action potential signals;

[0023] Fusion of the extracted features;

[0024] Generate instructions for controlling the rehabilitation gloves through the fused information;

[0025] Apply control instructions to the rehabilitation glove in real time.

[0026] Furthermore, the preprocessing of the monitored pressure signal and muscle action potential signal includes denoising, filtering, data alignment and normalization of the monitored pressure signal and muscle action potential signal;

[0027] The extracting of features from the pressure signal and the muscle action potential signal includes extracting finger pressure features from the pressure signal and extracting features related to hand muscle activity from the muscle action potential signal; the finger pressure features include maximum pressure and average pressure;

[0028] The fusing of the extracted features includes fusing the finger pressure features and features related to hand muscle activity using a multimodal neural network;

[0029] The generating of the control instructions of the rehabilitation gloves through the fused information includes taking the fused features as the input of the neural network model to obtain the control instructions of the rehabilitation gloves.

[0030] Furthermore, the low-frequency electrical stimulation interface supports an external conductive patch for electrical stimulation rehabilitation training.

[0031] Furthermore, the number of the servo linear motors is four, and the four servo linear motors respectively drive four fingers of the rehabilitation glove to move.

[0032] Furthermore, the control panel is an HMI control panel.

[0033] Furthermore, the battery is a lithium battery, and the wireless communication module is a USR-C322 WIFI module.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] The present invention provides a multi-source information fusion intelligent rehabilitation glove system, which integrates multi-source information sensory stimulation, including auditory, visual and tactile electrical stimulation, to enhance the rehabilitation process, especially to help the rehabilitation of damaged nerves. Secondly, the multi-source information fusion control module monitors and adjusts the patient's hand movements and muscle activities in real time through pressure sensors and electromyographic sensors, providing personalized rehabilitation support. In addition, the system introduces visual and auditory stimulation, such as LED lights and buzzers, during the rehabilitation process, making rehabilitation training more interesting and dynamic, and helping to improve the patient's rehabilitation experience.

[0036] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention and implement it according to the contents of the specification, the following is a detailed description of the preferred embodiments of the present invention in conjunction with the accompanying drawings. The specific implementation of the present invention is given in detail by the following embodiments and their accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0038] Figure 1 This is a block diagram of the multi-source information fusion intelligent rehabilitation glove system of Example 1;

[0039] Figure 2 This is the multi-source fusion control flow chart of Example 1. DETAILED DESCRIPTION

[0040] The present invention is further described below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that, under the premise of no conflict, the various embodiments or technical features described below can be arbitrarily combined to form a new embodiment.

[0041] Example 1

[0042] A multi-source information fusion intelligent rehabilitation glove system improves the rehabilitation effect by combining multiple sensory stimulation and fusion control, providing new possibilities for the rehabilitation of damaged nerves. Figure 1 As shown, the system includes a main control unit, a charging interface, a control panel, several servo linear motors, batteries, LED lights, buzzers, multi-source fusion control modules, wireless communication modules, and low-frequency electrical stimulation interfaces; wherein,

[0043] The main control unit is used to coordinate and control various parts of the rehabilitation gloves; in this embodiment, the main control unit adopts an STM32 main control board.

[0044] The charging port is used to charge the battery to ensure long-term operation of the system;

[0045] The control panel is used to provide an interactive interface between the user and the system, including mode selection, start and stop control, etc.; in this embodiment, the control panel is an HMI control panel, and the HMI control panel is used to adjust and configure the stimulation mode and frequency.

[0046] The servo linear motor is used to control the movement and position feedback of the rehabilitation glove. In this embodiment, there are four servo linear motors, which respectively drive the four fingers of the rehabilitation glove to move.

[0047] The battery is used to provide power to the system, so that the glove can be used without connecting cables; in this embodiment, the battery is a lithium battery;

[0048] LED lights are installed on the far end of the fingers on the back of the hand of the rehabilitation glove for visual stimulation; for example, the LED lights are controlled by the STM32 main control board and can flash green during rehabilitation training to provide visual stimulation. The specific stimulation mode and frequency can be adjusted and configured through the HMI control panel.

[0049] The buzzer is installed on the STM32 main control board for audio stimulation; for example, the buzzer plays musical notes to provide audio stimulation; the user can select the musical note mode through the HMI control panel and hear the corresponding music during rehabilitation training.

[0050] The low-frequency electrical stimulation interface supports external conductive stickers for electrical stimulation rehabilitation training; for example, the electrical stimulation mode is selected through the HMI control panel, and two electrode stickers are respectively attached to the forearm position of the extensor muscle group that controls hand extension and the flexor muscle group that controls hand flexion to perform low-frequency electrical stimulation rehabilitation.

[0051] The multi-source fusion control module integrates various sensors and control modules to comprehensively control and coordinate various aspects of the system. Specifically, it is used to monitor the pressure signal of the glove, monitor the muscle action potential signal, and control the movement and strength of the glove in real time according to the monitored pressure sensing signal and muscle action potential signal;

[0052] Wireless communication is used to connect to a network or other external devices to achieve data transmission and remote monitoring; in this embodiment, the wireless communication module is a USR-C322 WIFI module.

[0053] In one embodiment, when the user performs a specific rehabilitation action, the LED light can display corresponding guidance information, such as training instructions for the target finger. At the same time, the buzzer can provide real-time audio feedback to inform the user whether the target finger is being trained. If the user stops training, the buzzer will also stop. Low-frequency electrical stimulation can provide muscle stimulation according to the needs of the action to help the user complete the exercise. For example, when flexing the fingers, low-frequency electrical stimulation of the flexor muscles of the forearm can help the user better complete the training of flexing the fingers.

[0054] This embodiment provides a synergistic effect of multi-sensory stimulation by comprehensively utilizing LED lights, buzzers, and low-frequency electrical stimulation, integrating visual, auditory, and tactile sensations to provide more comprehensive and effective support for rehabilitation training; it can enhance the user's rehabilitation experience, increase the attractiveness and participation of rehabilitation training, and make it more interesting and interactive. In summary, the multi-source sensory feedback solution adds innovation and diversity to rehabilitation gloves, and is expected to improve rehabilitation effects, making it an important innovation in the medical and rehabilitation fields.

[0055] In this embodiment, the multi-source fusion control module includes a pressure sensor and an electromyographic sensor. The pressure sensor is used to monitor the pressure signal of the glove, and the electromyographic sensor is used to monitor the muscle activity potential signal. The pressure sensor adopts a resistive film pressure sensor. The multi-source fusion control module performs rehabilitation training control according to the hand pressure, and controls the movement and strength of the glove in real time to meet the needs of rehabilitation training.

[0056] like Figure 2 As shown, the multi-source fusion control module controls the movement and strength of the glove in real time according to the monitored pressure sensing signal and muscle action potential signal, including the following steps:

[0057] The monitored pressure signals and muscle action potential signals are preprocessed, including denoising, filtering, data alignment and normalization of the monitored pressure signals and muscle action potential signals to ensure consistent data quality. For example, the wavelet transform is used to decompose the signal into sub-signals of different scales. The wavelet transform has good local properties and can better adapt to the non-stationary nature of the signal. In the wavelet domain, noise is usually expressed as high-frequency components, while the signal is usually expressed as low-frequency components. Therefore, by selecting an appropriate wavelet basis function, the noise can be filtered out and the main information of the signal can be retained. The SG filtering algorithm (Savitzky-Golay filtering algorithm) can be used to smooth the signal. The dynamic time warping function can be used to align the signal, and the optimal alignment path between the two signals can be found by dynamic programming so that the alignment error on the path is minimized, which can solve the problems of time offset, scaling and shape difference between the two signals. The amplitude range of the signal can be mapped to a specified range, such as [0,1], by the signal normalization processing formula. Among them, the signal normalization processing formula is:

[0058] Normalized signal value = (original signal value - minimum value) / (maximum value - minimum value)

[0059] Where min and max are the minimum and maximum amplitudes of the signal, respectively.

[0060] Extracting features from pressure signals and muscle action potential signals includes extracting finger pressure features from pressure signals and extracting features related to hand muscle activity from muscle action potential signals; finger pressure features include maximum pressure, average pressure, etc.; features related to hand muscle activity include the average value of the absolute value of the signal amplitude, the square root of the amplitude mean, the number of times the waveform crosses a horizontal baseline, the cumulative sum of the waveform length, the number of times the sign of the amplitude change rate changes, the median frequency, the average value of the power frequency distribution, etc.; wherein, the average value of the absolute value of the signal amplitude is used to evaluate the strength and fatigue of muscle contraction, the square root of the amplitude mean is used to evaluate the strength and fatigue of muscle contraction, the number of times the waveform crosses a horizontal baseline is used to evaluate the contraction and relaxation state of the muscle, the cumulative sum of the waveform length is used to represent the complexity of the surface electromyography signal, the number of times the sign of the amplitude change rate changes is used to represent the state of the surface electromyography signal that is about to fluctuate, the median frequency is used to evaluate the strength and fatigue of muscle contraction, and the average value of the power frequency distribution is used to evaluate the strength and fatigue of muscle contraction. In order to match the time domain and frequency domain features of the signal and overcome the limitations of a single domain, time-frequency analysis technology is used to combine the two well. For example, short-time Fourier transform, S-transform (an extension of morlet wavelet transform), matching pursuit, etc. can be used. Taking short-time Fourier transform as an example, short-time Fourier transform is a windowing process based on Fourier transform, which improves the processing ability of non-stationary signals and strengthens the feature extraction ability.

[0061]

[0062] Where x(m) is the input signal; w(m) is the window function; it is flipped in time and has an offset of n samples; X(n,w) is a two-dimensional function defined in samples (time) and frequency.

[0063] The extracted features are fused, including the use of a multimodal neural network to fuse the finger pressure features and the features related to the hand muscle activity; it should be noted that the finger pressure features and the features related to the hand muscle activity can also be fused by weight fusion and feature-level fusion methods. Among them, the multimodal neural network mainly includes two parts: modal feature encoding and modal feature fusion. Modal feature encoding converts the perceptual information of different modalities into vector representations, usually using models such as convolutional neural networks (CNN), recurrent neural networks (RNN) or transformers for encoding. Modal feature fusion integrates the feature vectors of different modalities to improve the performance and stability of the model while retaining the features of different modalities.

[0064] The fused information is used to generate instructions for controlling the rehabilitation gloves, including using the fused features as the input of the neural network model to obtain the control instructions for the rehabilitation gloves; it should be noted that accurate rehabilitation control can also be achieved based on the fused information through machine learning algorithms or other control strategies. The fusion method and algorithm in actual use may vary depending on the specific system and rehabilitation goals. By integrating information from different sensors, more accurate rehabilitation glove control can be achieved to meet the rehabilitation needs of patients. In this embodiment, the neural network model is trained by focusing on the fusion features of the finger pressure features corresponding to the pressure signal and the features related to the hand muscle activity corresponding to the muscle activity potential signal, and the control instructions corresponding to the intention of the user's finger activity are learned to obtain the optimal neural network model, and the actual fused information is used as the input of the optimal neural network model to obtain the control instructions corresponding to the intention of the user's finger activity, which can be used to control a single finger of the rehabilitation glove.

[0065] Control instructions are applied to rehabilitation gloves in real time to meet the needs of rehabilitation training. At the same time, the patient's hand activities can be monitored in real time and adjusted when necessary.

[0066] The present invention provides a multi-source information fusion intelligent rehabilitation glove system, which aims to enhance the rehabilitation process by integrating multi-sensory stimulation, including auditory, visual and tactile electrical stimulation, and multi-source information fusion control, including pressure and electromyographic signals, and in particular provides support in the rehabilitation of damaged nerves. It has a wide range of application potentials, including rehabilitation therapy and rehabilitation training, provides a new way for the rehabilitation of damaged nerves, and plays a key role in the medical and health fields.

[0067] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0068] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0069] The above are only embodiments of this specification and are not intended to limit one or more embodiments of this specification. For those skilled in the art, one or more embodiments of this specification may have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of this specification shall be included in the scope of the claims of one or more embodiments of this specification.

Claims

1. A multi-source information fusion intelligent rehabilitation glove system, characterized by: It includes a main control unit, a charging interface, a control panel, several servo linear motors, batteries, LED lights, buzzers, multi-source fusion control modules, wireless communication modules, and low-frequency electrical stimulation interfaces; among them, The main control unit is used to coordinate and control various parts of the rehabilitation glove; The charging interface is used to charge the battery; The control panel is used to provide an interactive interface between the user and the system; The servo linear motor is used to control the movement and position feedback of the rehabilitation glove; The battery is used to provide power to the system; The LED light is installed at the distal end of the fingers in the back of the hand direction of the rehabilitation glove for visual stimulation; The buzzer is used for audio stimulation; The low-frequency electrical stimulation interface is used for electrical stimulation rehabilitation training; The multi-source fusion control module is used to monitor the pressure signal of the glove and the muscle action potential signal, and to control the movement and strength of the glove in real time according to the monitored pressure sensing signal and muscle action potential signal; The wireless communication is used for data transmission and remote monitoring.

2. The multi-source information fusion intelligent rehabilitation glove system according to claim 1, characterized in that: The control panel is used to adjust and configure stimulation mode and frequency.

3. The multi-source information fusion intelligent rehabilitation glove system according to claim 1, characterized in that: The multi-source fusion control module includes a pressure sensor and an electromyographic sensor. The pressure sensor is used to monitor the pressure signal of the glove, and the electromyographic sensor is used to monitor the muscle activity potential signal.

4. The multi-source information fusion intelligent rehabilitation glove system according to claim 3, characterized in that: The pressure sensor is a resistive thin film pressure sensor.

5. The multi-source information fusion intelligent rehabilitation glove system according to claim 1 or 3, characterized in that: The multi-source fusion control module controls the movement and strength of the glove in real time according to the monitored pressure sensing signal and muscle action potential signal, including the following steps: Pre-process the monitored pressure signals and muscle action potential signals; Extract features from pressure signals and muscle action potential signals; Fusion of the extracted features; Generate instructions for controlling the rehabilitation gloves through the fused information; Apply control instructions to the rehabilitation glove in real time.

6. The multi-source information fusion intelligent rehabilitation glove system according to claim 5, characterized in that: The preprocessing of the monitored pressure signal and muscle action potential signal includes denoising, filtering, data alignment and normalization of the monitored pressure signal and muscle action potential signal; The extracting of features from the pressure signal and the muscle action potential signal includes extracting finger pressure features from the pressure signal and extracting features related to hand muscle activity from the muscle action potential signal; The finger pressure characteristics include maximum pressure and average pressure; The fusing of the extracted features includes fusing the finger pressure features and features related to hand muscle activity using a multimodal neural network; The generating of the control instructions of the rehabilitation gloves through the fused information includes taking the fused features as the input of the neural network model to obtain the control instructions of the rehabilitation gloves.

7. The multi-source information fusion intelligent rehabilitation glove system according to claim 1, characterized in that: The low-frequency electrical stimulation interface supports external conductive patches for electrical stimulation rehabilitation training.

8. The multi-source information fusion intelligent rehabilitation glove system according to claim 1, characterized in that: The number of the servo linear motors is four, and the four servo linear motors respectively drive four fingers of the rehabilitation glove to move.

9. The multi-source information fusion intelligent rehabilitation glove system according to claim 1, characterized in that: The control panel is an HMI control panel.

10. The multi-source information fusion intelligent rehabilitation glove system according to claim 1, characterized in that: The battery is a lithium battery, and the wireless communication module is a USR-C322 WIFI module.

Images