Hand rehabilitation training method and device based on emotion feedback, equipment and medium

Abstract

The invention provides a hand rehabilitation training method, device and equipment based on emotion feedback and a medium, and relates to the technical field of intelligent medical treatment. The specific implementation mode comprises the steps that in the process of hand rehabilitation training of a target patient, the emotional state of the patient is monitored, emotional monitoring data of the patient is obtained, and emotional features are extracted from the emotional monitoring data; inputting the sentiment features into a sentiment classification model to obtain a current sentiment state category of the patient; the sentiment classification model is a classification model pre-constructed based on a support vector machine; and dynamically adjusting the current hand rehabilitation training scheme of the patient according to the current emotional state category of the patient. The problems that in the prior art, emotion monitoring and feedback are lacked, the training mode is single, and the patient participation degree is low are solved, an emotion recognition and feedback mechanism is fused, a personalized hand rehabilitation training scheme is intelligently recommended and dynamically adjusted according to the emotion state and the rehabilitation progress of the patient, and the rehabilitation effect and the patient positive degree are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, in particular to the technical field of smart medical treatment, and especially to a hand rehabilitation training method and device based on emotional feedback, equipment and a medium. BACKGROUND

[0002] Hand rehabilitation training is of great significance in the fields of neurological rehabilitation and work injury recovery. The current advanced hand rehabilitation training system is mainly based on virtual reality (VR) or augmented reality (AR) technology, combined with motion capture and biofeedback, to provide an immersive training environment for patients. For example, some products use motion capture technology to monitor the patient's hand movements in real time and feed the data back to the system, which adjusts the training task difficulty according to a preset algorithm. The training scheme is mainly based on physiological indicators such as hand strength and joint mobility, and gradually increases the training intensity in a progressive resistive exercise mode.

[0003] However, the existing hand rehabilitation training focuses more on physiological function recovery, often ignoring the psychological state and emotional needs of patients during the rehabilitation process, which may affect the rehabilitation effect and patient compliance. Moreover, the existing training scheme is based on physiological indicators and has a fixed training mode, which is difficult to adapt to the differences in training state caused by changes in patients' emotions. In addition, in the existing training scheme, patients are mostly passive in accepting tasks, lacking a mechanism for actively adjusting the training process, which is not conducive to improving the rehabilitation enthusiasm. SUMMARY

[0004] To solve the problems of lack of emotional monitoring and feedback, single training mode and low patient participation in the prior art, a hand rehabilitation training method and device based on emotional feedback, equipment and a medium are provided.

[0005] According to a first aspect, a hand rehabilitation training method based on emotional feedback is provided, the method comprising: monitoring the emotional state of a target patient during the hand rehabilitation training of the target patient, obtaining emotional monitoring data of the target patient, and extracting emotional features from the emotional monitoring data; inputting the emotional features into an emotional classification model to obtain the emotional state category of the target patient; wherein the emotional classification model is a classification model pre-constructed based on a support vector machine; dynamically adjusting the hand rehabilitation training scheme of the target patient according to the emotional state category of the target patient. According to a second aspect, a hand rehabilitation training device based on emotional feedback is provided, comprising: The data acquisition and feature extraction module is configured to monitor the emotional state of the target patient during the hand rehabilitation training of the target patient, obtain emotional monitoring data of the target patient, and extract emotional features from the emotional monitoring data. The emotion classification module is configured to input the emotional features into an emotion classification model to obtain the emotional state category of the target patient. The emotion classification model is a classification model pre-constructed based on a support vector machine. The training scheme adjustment module is configured to dynamically adjust the current hand rehabilitation training scheme of the target patient according to the emotional state category of the target patient.

[0006] According to a third aspect, an electronic device is provided, comprising: one or more processors; a storage device configured to store one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of the embodiments of the hand rehabilitation training method based on emotional feedback.

[0007] According to a fourth aspect, a computer-readable storage medium is provided, which stores a computer program, when the computer program is executed by a processor, the method of any one of the embodiments of the hand rehabilitation training method based on emotional feedback is implemented.

[0008] According to the scheme of the present application, the emotional fluctuation of the patient during the rehabilitation training is monitored in real time, and the emotional features of the patient are extracted through the emotional state monitoring data of the patient. The emotional features are classified through an emotional classification model to obtain the emotional state category of the patient during the training. Then, the current training scheme is adjusted according to the obtained emotional state category, so as to fuse the current emotional state of the patient with the training scheme, form an emotional recognition and feedback mechanism of the patient, intelligently recommend and dynamically adjust the personalized hand rehabilitation training scheme according to the emotional state of the patient and the rehabilitation progress, and improve the rehabilitation effect and the patient's enthusiasm. BRIEF DESCRIPTION OF DRAWINGS Other features, objects and advantages of the present application will become more apparent from the following detailed description of non-limiting embodiments, made with reference to the following drawings: Figure 1 is an exemplary system architecture diagram to which some embodiments of the present application can be applied; Figure 2 is a flowchart of one embodiment of the hand rehabilitation training method based on emotional feedback according to the present application; Figure 3 is a structural schematic diagram of one embodiment of the hand rehabilitation training device based on emotional feedback according to the present application; Figure 4is a structural schematic diagram of another embodiment of a hand rehabilitation training device based on emotional feedback according to the present application; Figure 5 is a block diagram of an electronic device for implementing a hand rehabilitation training method based on emotional feedback according to an embodiment of the present application. DETAILED DESCRIPTION

[0009] Exemplary embodiments of the present application are described herein with reference to the accompanying drawings, in which various details of the embodiments of the present application are set forth to assist in the understanding of the present application. It will be apparent to those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the spirit and scope of the present application. Also, the description herein is merely illustrative of the principles of the present application and does not limit the scope of the present application. Therefore, the scope of the present application should be defined by the appended claims.

[0010] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0011] Figure 1 An exemplary system architecture 100 to which the hand rehabilitation training method based on emotional feedback or the hand rehabilitation training device based on emotional feedback according to the present application can be applied is shown.

[0012] As shown in Figure 1 , the system architecture 100 can include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a communication link medium between the terminal devices 101, 102, 103 and the server 105. The network 104 can include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0013] The user can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the terminal devices 101, 102, 103, such as video applications, live applications, instant messaging tools, email clients, social platform software, etc.

[0014] The terminal devices 101, 102, and 103 herein can be hardware or software. When the terminal devices 101, 102, and 103 are hardware, they can be various electronic devices with a display screen, including but not limited to a smart phone, a tablet computer, an electronic book reader, a laptop computer, a desktop computer, and the like. When the terminal devices 101, 102, and 103 are software, they can be installed in the above-listed electronic devices. They can be implemented as multiple software or software modules (for example, multiple software or software modules for providing distributed services) or as a single software or software module. No specific limitation is made herein.

[0015] The server 105 can be a server providing various services, for example, a background server providing support for the terminal devices 101, 102, and 103. The background server can classify the current emotional state of a patient according to the emotional features of the patient, and feed back the processing result (for example, the emotional state category) to the terminal device.

[0016] It should be noted that the hand rehabilitation training method based on emotional feedback provided by the embodiments of the present application can be executed by the server 105 or the terminal devices 101, 102, and 103, and accordingly, the hand rehabilitation training device based on emotional feedback can be arranged in the server 105 or the terminal devices 101, 102, and 103.

[0017] It should be understood that Figure 1 The number of terminal devices, networks, and servers in

[0018] With reference to Figure 2 , a flow 200 of one embodiment of the hand rehabilitation training method based on emotional feedback according to the present application is shown. The hand rehabilitation training method based on emotional feedback includes the following steps: Step 201, in the process of hand rehabilitation training of a target patient, the emotional state of the target patient is monitored, the emotional monitoring data of the target patient is obtained, and emotional features are extracted from the emotional monitoring data; Step 202, the emotional features are input into an emotional classification model to obtain the emotional state category of the target patient at present; wherein the emotional classification model is a classification model pre-constructed based on a support vector machine; Step 203, according to the emotional state category of the target patient at present, the current hand rehabilitation training scheme of the target patient is dynamically adjusted.

[0019] Since the existing hand rehabilitation training scheme lacks emotional monitoring and feedback, and the training mode is single, it leads to low patient participation and poor training effect. Based on this, in the embodiment, the hand rehabilitation training method based on emotional feedback runs on the execution subject (such as the server or terminal device shown in Figure 1 The closed-loop feedback system of emotional perception-machine learning classification-training parameter dynamic mapping is constructed, the emotional fluctuations of the patient are captured through physiological signals (such as heart rate variability, skin conductance) or behavior data (such as facial expression, voice tone), and the limitation that the traditional rehabilitation training scheme only focuses on single motor function is broken. The rehabilitation training scheme can be optimized according to the real-time emotional state of the patient to improve the adaptability and training effect of the training. In addition, the dynamic adjustment of the rehabilitation training scheme can timely respond to the emotional changes of the patient, avoid the influence of negative emotions on the rehabilitation process, and improve the enthusiasm and cooperation of the patient through emotional perception, thereby promoting the rehabilitation effect.

[0020] In the process of rehabilitation training of the patient, in order to accurately capture the emotional fluctuations of the patient, the physiological signals or behavior data of the patient in multiple aspects need to be obtained. In some optional implementation manners of the embodiment, the emotional state of the target patient is monitored, the emotional monitoring data of the patient is obtained, and emotional features are extracted from the emotional monitoring data, including: the emotional state of the target patient in multiple aspects is monitored in real time, and multiple emotional monitoring data of the patient are obtained respectively; the multiple emotional monitoring data are respectively subjected to emotional feature extraction, multiple emotional features are obtained, and the multiple emotional features are subjected to feature vector fusion to obtain a comprehensive feature vector.

[0021] In specific implementation, the emotional data of the patient, such as facial expression image data, voice tone data, physiological signal monitoring data, etc. are collected in real time during the training process. The specific process includes the following contents: (1) Facial expression image acquisition: a high-definition camera can be installed in front of the patient at a suitable position to ensure that the patient's entire facial expression can be clearly captured. The camera is turned on to continuously collect the patient's facial image data at a frequency of 30 frames per second, and the data is stored as an image sequence file in JPEG format with a resolution of 1920x1080.

[0022] (2) Voice tone data acquisition: a professional microphone can be placed near the patient's training environment, about 0.5 meters away from the patient, to ensure clear voice pickup. The microphone collects the patient's voice tone data in real time at a sampling frequency of 44.1 kHz, and stores it as a WAV format audio file.

[0023] (3) Physiological signal monitoring: Wearable physiological signal sensors are worn by the patient, including a heart rate strap and a skin conductance sensor. The heart rate strap is attached to the patient's chest to monitor ECG signals in real time to obtain accurate heart rate data, which is transmitted to the data acquisition system at a frequency of 100 sample points per second; the skin conductance sensor is worn on the patient's finger to collect skin conductance signals reflecting the patient's sweating, which is also transmitted at a frequency of 100 sample points per second.

[0024] Next, feature extraction is performed on the facial expression image, voice tone data, and physiological signal respectively. The specific process includes the following contents: (1) Facial expression feature extraction The collected facial expression images are subjected to grayscale processing: the color facial images are converted to grayscale images to reduce data dimension and computational complexity, and the conversion formula is as follows: Gray = 0.299R + 0.587G + 0.114B Where R: red channel value, G: green channel value, B: blue channel value.

[0025] Based on the obtained grayscale image, face detection is performed: a face detection algorithm based on deep learning (such as Multi Task Cascaded Convolutional Networks, MTCNN) is used to locate the facial region and extract the facial image. MTCNN includes three stages: Proposal Network (P-Net): generates initial facial candidate regions.

[0026] Refine Network (R-Net): filters and refines the candidate regions.

[0027] Output Network (O-Net): further refines the facial region and performs facial key point detection.

[0028] Then perform facial expression feature extraction: use a convolutional neural network (CNN) to extract features from the grayscale facial image. Assuming that the input image is X, the convolution kernel is W, the bias term is b, and the activation function is ReLU, then the feature map F after convolution operation is:

[0029] Where * represents convolution operation, W represents convolution kernel, and b represents bias term.

[0030] After multiple convolution layers and pooling layers, the final facial expression feature vector F is obtained, with a dimension of 128.

[0031] (2) Voice tone feature extraction First, the collected voice tone data is preprocessed: the voice signal is preprocessed, including removing background noise, endpoint detection, etc. The preprocessing steps are as follows: Remove background noise: use spectral subtraction to remove background noise. Assume that the original voice signal is x(t), the background noise is n(t), and the denoised voice signal y(t) is:

[0032] Then perform voice endpoint detection: use short-time energy and short-time zero-crossing rate to detect the endpoint of the voice signal.

[0033] Next, the Mel Frequency Cepstral Coefficient (MFCC) extraction method is used to extract important acoustic features from the voice signal. The MFCC extraction process mainly includes the following steps: preprocessing, Fourier Transform (FFT), Mel filter bank processing, logarithmic compression, and Discrete Cosine Transform.

[0034] Among them, the short-time Fourier transform (STFT): the preprocessed voice signal is divided into multiple frames, each frame length is N, and the frame shift is M. Fourier transform is performed on each frame to obtain the frequency spectrum :

[0035] Among them, represents the nth sample, represents the window function (such as the Hanning window) Mel filter bank: map the frequency spectrum to the Mel scale and use the Mel filter bank to extract features. The frequency range of the Mel filter bank is from to , usually divided into N filters.

[0036] Discrete Cosine Transform (DCT): perform discrete cosine transform on the output of the Mel filter bank to obtain the MFCC coefficient C:

[0037] Among them, represents the output of the Mel filter bank, and k represents the serial number of the MFCC coefficient.

[0038] Finally, a 13-dimensional MFCC feature vector is obtained.

[0039] (3) Physiological signal feature extraction Heart rate variability (HRV) analysis: HRV refers to the natural variation of the time interval (RR interval) between consecutive heartbeats. This variation reflects the small adjustments of the heart rhythm, mainly regulated by the interaction of the sympathetic and parasympathetic nervous systems.

[0040] RR interval extraction: Extract the RR interval sequence from the electrocardiogram signal { , ,…, }.

[0041] RMSSD calculation: Calculate the root mean square of adjacent RR interval differences (RMSSD):

[0042] Skin conductance level (SCL) analysis: SCL analysis is a physiological monitoring technique that assesses sympathetic nervous system (SNS) activity by measuring changes in the skin's surface conductance. It provides objective quantitative indicators of emotional arousal, stress response, and cognitive load by capturing sympathetic nerve-driven sweat gland activity. SCL is primarily regulated by electrolytes secreted by sweat glands, such as sodium and chloride ions. For example, when the sympathetic nervous system is excited, cholinergic nerve endings release acetylcholine, which activates sweat glands to secrete and increase skin conductivity.

[0043] First, filter the skin conductance signal: Filter the skin conductance signal to remove noise.

[0044] Then calculate SCL: Calculate the average skin conductance over a period of time: Where G(t) represents the skin conductance signal and T represents the time interval.

[0045] In this embodiment, after obtaining multiple emotional features, the multiple emotional features are fused into a feature vector to obtain a comprehensive feature vector. By fusing the feature vector, it is matched with the patient training scheme, thereby improving the training effect.

[0046] The construction process of the comprehensive feature vector specifically includes: Fusing the facial expression feature vector (128 dimensions), the speech tone MFCC feature vector (13 dimensions), and the physiological signal features (RMSSD and SCL, 2 dimensions) to construct a 143-dimensional comprehensive feature vector V:

[0047] The obtained comprehensive feature vector V is input into the pre-constructed emotion classification model to obtain an emotion classification result, i.e., an emotion state category. This embodiment dynamically adjusts the individualized training scheme by constructing an emotion classification model to improve the training effect.

[0048] In specific implementation, the emotion classification model is a support vector machine (SVM) model: The goal of SVM is to find a hyperplane that separates data of different categories. The optimization problem is represented as:

[0049] The constraint condition is:

[0050] wherein w represents a weight vector, C represents a regularization parameter, represents a category label (emotional state), represents an input feature vector, represents a feature mapping function, and b represents a bias term, represents a slack variable.

[0051] Kernel function selection: radial basis function (RBF) is selected as the kernel function of SVM:

[0052] wherein γ represents a kernel function parameter.

[0053] In order to improve the accuracy of the emotional classification model, in some optional implementation manners of the embodiment, the method further comprises: using sample data labeled with emotional category labels, optimizing the model parameters of the emotional classification model according to the grid search method, training the emotional classification model using the optimized model parameters, and obtaining the trained emotional classification model.

[0054] In specific implementation, the training process of the model comprises: (1) Data preparation: a large amount of sample data labeled with emotional categories is collected, each sample comprising a comprehensive feature vector V and a corresponding emotional label y.

[0055] (2) Parameter optimization: the parameters C and γ of the SVM model are optimized using the grid search method. The best parameter combination is determined through cross-validation.

[0056] (3) Model training: the SVM model is trained using the optimized parameters, and a trained emotional classification model is obtained.

[0057] In some optional implementation manners of the embodiment, the current hand rehabilitation training scheme of the patient is dynamically adjusted according to the current emotional state category of the patient, comprising: the training scheme difficulty, training duration and training rest interval of the patient are dynamically adjusted according to the current emotional state category of the patient; wherein the emotional state category comprises an anxiety state and a low mood state.

[0058] In specific implementation, in the rehabilitation training process of the patient, data is collected in real time and a comprehensive feature vector V is generated, which is input into the trained SVM model, and the model outputs the current emotional state category of the patient , according to the emotional state category Adjust the training scheme. Specifically, the training scheme difficulty, training duration, and training rest interval can be adjusted according to the emotional state of the patient.

[0059] wherein (1) when the patient is in an anxious state, the difficulty of the training scheme is automatically reduced to reduce the patient's stress and enable them to better focus on the training task. When the anxiety degree A exceeds the threshold value , the training scheme difficulty is adjusted by the following formula:

[0060] wherein, is the adjusted training scheme difficulty, is the initial difficulty, with a value range of [0, 1], and γ is the adjustment coefficient, with a value range of [0, 1], is the anxiety degree threshold value, with a value range of [0, 1], and A is the current anxiety degree, with a value range of [0, 1].

[0061] (2) When the patient is in an anxious state, the training rest interval of the patient is increased, giving the patient more time to relax. The training rest interval is adjusted by the following formula:

[0062] wherein, is the adjusted training rest interval (seconds), is the initial training rest interval (seconds), is the adjustment coefficient (seconds), is the anxiety degree threshold value, with a value range of [0, 1], and A is the current anxiety degree, with a value range of [0, 1].

[0063] For example, the initial training rest interval = 15 seconds, the current anxiety degree A = 0.8, the anxiety degree threshold = 0.7, and the adjustment coefficient = 5 seconds, then the new training rest interval is:

[0064] (3) When the patient is in a depressed state, the training duration is shortened to avoid excessive fatigue or resistance from the patient. The training duration is adjusted by the following formula:

[0065] wherein, is the adjusted training duration (minutes), is the initial training duration (minutes), and δ is the adjustment coefficient, with a value range of [0, 1], The mood depression threshold value is in the range of [0, 1], S is the current mood depression degree, and the value range is [0, 1].

[0066] For example, the initial training duration is 30 minutes, the current mood depression degree S of the patient is 0.7, the mood depression threshold value is 0.6, and the adjustment coefficient δ is 0.3. Then the new training duration is:

[0067] The embodiment combines emotion recognition and feedback mechanism, maps the collected emotion data features to the hand rehabilitation training model parameters, intelligently recommends and dynamically adjusts the personalized hand rehabilitation training scheme according to the patient's emotional state and rehabilitation progress, and improves the rehabilitation effect and patient's enthusiasm.

[0068] Further referring to Figure 3 , as an implementation of the method shown in the above figures, the application provides an embodiment of a hand rehabilitation training device based on emotional feedback. The device embodiment corresponds to the method embodiment shown in Figure 2 . In addition to the features described below, the device embodiment can also include the same or corresponding features or effects as the method embodiment shown in Figure 2 . The device can be applied to various electronic devices.

[0069] As shown in Figure 3 , the hand rehabilitation training device based on emotional feedback 300 of the embodiment includes a data acquisition and feature extraction module 301, an emotion classification module 302, and a training scheme adjustment module 303. The data acquisition and feature extraction module 301 is used to monitor the emotional state of the target patient during hand rehabilitation training, obtain emotional monitoring data of the target patient, and extract emotional features from the emotional monitoring data. The emotion classification module 302 is used to input the emotional features into an emotion classification model to obtain the current emotional state category of the target patient. The emotion classification model is a classification model pre-constructed based on a support vector machine. The training scheme adjustment module 303 is used to dynamically adjust the current hand rehabilitation training scheme of the target patient according to the current emotional state category of the target patient.

[0070] In the embodiment, the specific processing of the data acquisition and feature extraction module 301, the emotion classification module 302, and the training scheme adjustment module 303 of the hand rehabilitation training device based on emotional feedback 300 and the technical effects brought by them can be respectively referred to the related descriptions of step 201, step 202, and step 203 in the corresponding embodiment, which will not be repeated here. Figure 2 ​​​

[0071] In some optional implementations of this embodiment, the data acquisition and feature extraction module 301 monitors various aspects of the target patient's emotional state in real time to obtain various types of emotion monitoring data; performs emotion feature extraction on each of the various types of emotion monitoring data to obtain various emotion features; and fuses the various emotion features into feature vectors to obtain a comprehensive feature vector. The various types of emotion monitoring data include the patient's facial expression image data, voice and intonation data, and physiological signal monitoring data.

[0072] In some optional implementations of this embodiment, the training program adjustment module 303 dynamically adjusts the patient's training program difficulty, training duration and training rest interval according to the patient's current emotional state category; wherein the emotional state category includes anxiety state and depression state.

[0073] In some optional implementations of this embodiment, such as Figure 4 As shown, the hand rehabilitation training device 300 based on emotion feedback also includes a model construction and training module 304. The model construction and training module 304 is used to construct an emotion classification model, and use sample data marked with emotion category labels to optimize the model parameters of the emotion classification model according to the grid search method, and use the optimized model parameters to train the emotion classification model to obtain the trained emotion classification model.

[0074] The present embodiment provides a hand rehabilitation training device 300 based on emotional feedback, which monitors the patient's emotional fluctuations during rehabilitation training in real time, extracts the patient's emotional characteristics through the patient's emotional state monitoring data, classifies the emotional characteristics through an emotional classification model, obtains the patient's emotional state category during training, and then adjusts the current training program based on the obtained emotional state category. The device includes a data acquisition and feature extraction module 301, an emotional classification module 302, a training program adjustment module 303, and a model construction and training module 304, thereby integrating the patient's current emotional state with the training program, forming a patient's emotional recognition and feedback mechanism, and intelligently recommending and dynamically adjusting personalized hand rehabilitation training programs based on the patient's emotional state and rehabilitation progress, thereby improving rehabilitation effects and patient enthusiasm.

[0075] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium. Figure 5, is a block diagram of an electronic device for a hand rehabilitation training method based on emotional feedback according to an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0076] like Figure 5 As shown, the electronic device includes: one or more processors 401, a memory 402, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 5 A processor 401 is taken as an example.

[0077] Memory 402 is the non-transitory computer-readable storage medium provided in this application. The memory stores instructions executable by at least one processor to cause the at least one processor to perform the hand rehabilitation training method based on emotional feedback provided in this application. The non-transitory computer-readable storage medium of this application stores computer instructions for causing a computer to perform the hand rehabilitation training method based on emotional feedback provided in this application.

[0078] The memory 402 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as the program instructions / modules corresponding to the hand rehabilitation training method based on emotional feedback in the embodiment of the present application (for example, the attached Figure 4The illustrated data collection and feature extraction module 301, sentiment classification module 302, training scheme adjustment module 303, and model construction and training module 304). The processor 401 performs various functional applications and data processing of the server by running non-transitory software programs, instructions, and modules stored in the memory 402, i.e., implements the above-mentioned method embodiment of the hand rehabilitation training method based on emotional feedback.

[0079] The memory 402 can include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required by a function; the data storage area can store data created according to the use of the hand rehabilitation training electronic device based on emotional feedback, etc. In addition, the memory 402 can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory 402 can optionally include a memory disposed remotely with respect to the processor 401, which can be connected to the hand rehabilitation training electronic device based on emotional feedback through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0080] The electronic device of the hand rehabilitation training method based on emotional feedback can also include an input device 403 and an output device 404. The processor 401, the memory 402, the input device 403, and the output device 404 can be connected by a bus or other means, Figure 5 For example, by way of bus connection.

[0081] The input device 403 can receive input digital or character information, and generate key signal input related to user settings and function control of the hand rehabilitation training electronic device based on emotional feedback, such as touch screen, keypad, mouse, trackpad, touchpad, pointing stick, one or more mouse buttons, trackball, joystick, etc. The output device 404 can include a display device, an auxiliary lighting device (e.g., LED), and a tactile feedback device (e.g., a vibration motor), etc. The display device can include but is not limited to a liquid crystal display (LCD), a light-emitting diode (LED) display, and a plasma display. In some embodiments, the display device can be a touch screen.

[0082] Various implementations of the systems and techniques described herein can be realized in digital electronic circuitry, integrated circuitry, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpreted on a programmable system comprising at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0083] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for programmable processors and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., a magnetic disk, an optical disk, a memory, a programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0084] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0085] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0086] The computer system can include clients and servers. This description uses the terms "client" and "server" to describe the roles of the computers in the interactions. The computer with which the user interacts can be called a client device, and the other computer(s) that the user actually requests data from can be called servers.

[0087] The computer system can include clients and servers. This description uses the terms "client" and "server" to describe the roles of the computers in the interactions. The computer with which the user interacts can be called a client device, and the other computer(s) that the user actually requests data from can be called servers.

[0088] The units described in the embodiments of the present application can be implemented in the form of software, or can be implemented in the form of hardware. The described units can also be arranged in a processor, for example, can be described as: a processor includes a data acquisition and feature extraction module, an emotion classification module, a training scheme adjustment module. Among them, the name of these units does not constitute a limitation to the unit itself in some cases, for example, the emotion classification module can also be described as "model processing module".

[0089] As another aspect, the application also provides a computer readable medium, which can be included in the device described in the above embodiments, or can exist independently without being assembled into the device. The computer readable medium carries one or more programs, which, when executed by the device, cause the device to: fuse the current emotional state of the patient with the training scheme to form an emotional recognition and feedback mechanism of the patient, intelligently recommend and dynamically adjust the personalized hand rehabilitation training scheme according to the emotional state of the patient and the rehabilitation progress, and improve the rehabilitation effect and the enthusiasm of the patient.

[0090] The above description is merely the preferred embodiments of the application and the explanation of the principles of the applied technology. It should be understood by those skilled in the art that the scope of the application involved in the application is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by any combinations of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features can be replaced with the technical features disclosed in the application (but not limited to) having similar functions to form technical solutions.

Claims

1. A hand rehabilitation training method based on emotional feedback, characterized in that: The method comprises: During the hand rehabilitation training of the target patient, the emotional state of the target patient is monitored to obtain emotional monitoring data of the target patient, and emotional features are extracted from the emotional monitoring data; Inputting the emotional features into an emotional classification model to obtain the current emotional state category of the target patient; wherein the emotional classification model is a classification model pre-built based on a support vector machine; According to the current emotional state category of the target patient, the current hand rehabilitation training program of the target patient is dynamically adjusted.

2. The method according to claim 1, characterized in that Monitoring the emotional state of the target patient, obtaining emotional monitoring data of the target patient, and extracting emotional features from the emotional monitoring data, including: Real-time monitoring of the target patient's various emotional states to obtain various emotional monitoring data of the target patient; Emotional features are extracted from the multiple emotion monitoring data to obtain multiple emotion features, and feature vectors are fused on the multiple emotion features to obtain a comprehensive feature vector.

3. The method according to claim 1, characterized in that Dynamically adjusting the current hand rehabilitation training program of the target patient according to the current emotional state category of the target patient includes: According to the current emotional state category of the target patient, the difficulty, training duration and training rest interval of the target patient's training program are dynamically adjusted; wherein, the emotional state category includes anxiety state and depression state.

4. The method according to claim 3, characterized in that If the target patient's current emotional state is classified as an anxious state, the difficulty of the target patient's training program is reduced and / or the target patient's training rest interval is increased; wherein the difficulty of the training program is adjusted by the following formula: Where, The difficulty of the adjusted training program is is the initial difficulty, γ is the adjustment coefficient, is the anxiety threshold, A is the current anxiety level; The training rest interval is adjusted by the following formula: Where, For the adjusted training rest interval, For the initial training rest interval, is the adjustment coefficient, is the anxiety threshold, and A is the current anxiety level.

5. The method according to claim 3, characterized in that If the target patient's current emotional state is a depressed state, the training duration is shortened; wherein the training duration is adjusted by the following formula: Where, is the adjusted training duration, is the initial training time, δ is the adjustment coefficient, is the depression threshold, and S is the current depression level.

6. The method according to claim 2, characterized in that The multiple emotion monitoring data include facial expression image data, voice intonation data and physiological signal monitoring data of the target patient.

7. The method according to claim 1, characterized in that The method further comprises: Sample data marked with emotion category labels are used to optimize the model parameters of the emotion classification model according to the grid search method, and the emotion classification model is trained using the optimized model parameters to obtain the trained emotion classification model.

8. A hand rehabilitation training device based on emotional feedback, characterized in that: include: A data acquisition and feature extraction module is used to monitor the emotional state of the target patient during the hand rehabilitation training, obtain the target patient's emotional monitoring data, and extract emotional features from the emotional monitoring data; An emotion classification module, configured to input the emotion features into an emotion classification model to obtain the current emotion state category of the target patient; wherein the emotion classification model is a classification model pre-built based on a support vector machine; The training program adjustment module is used to dynamically adjust the current hand rehabilitation training program of the target patient according to the current emotional state category of the target patient.

9. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images