Music-guided hand tactile training method and system

Abstract

The present application relates to a music-guided hand tactile training method and system, which includes: detecting and generating an emotion recognition state; outputting a music data sequence, and changing the output speed of the music data sequence according to an adjustment parameter; converting the music data sequence into a finger movement sequence corresponding to the music data sequence, and using the finger movement sequence to stimulate the corresponding finger; detecting the finger movement within the time window when the stimulation is completed, and obtaining marked data after marking the finger movement sequence according to the finger movement amplitude, wherein the marked data includes completed data with a finger movement amplitude higher than expected, and unfinished data with a finger movement amplitude lower than expected; using the completed data, the unfinished data, and the emotion recognition state to change the adjustment parameter. The present application changes the output speed of the music data sequence in real time by adjusting the parameter, and performs real-time state evaluation and adjustment of the effect in combination with tactile feedback.

Description

Technical Field

[0001] The present application relates to the field of rehabilitation engineering, and in particular to a music-guided hand tactile training method and system. Background Art

[0002] Haptic interaction is based on the human tactile perception mechanism. Using tactile devices, it simulates the human perception of real objects, enabling human-computer interaction technology that perceives and reproduces tactile sensations. Emotion is a general term for a series of subjective cognitive experiences, representing psychological and physiological states resulting from the integration of multiple sensations, thoughts, and behaviors. Muscle memory refers to the fact that muscles have a memory effect. After repeated movements, the muscles form a conditioned reflex.

[0003] In the process of human understanding of the objective world, approximately 20% to 30% of information comes from non-visual perception, of which force touch is a significant source. Among the five sensory channels for humans to acquire environmental information, force touch is second only to vision and is the only channel capable of bidirectional information transmission. Force touch interaction is based on the human force touch perception mechanism. Using force touch devices, it simulates the human perception of real objects, enabling human-computer interaction technology that perceives and reproduces force touch.

[0004] Meta's Reality Labs tactile gloves can reproduce a series of tactile sensations in real life in the virtual world, including simulating the feeling of human hands touching material textures, pressure feedback, and vibration feedback.

[0005] Scientists at the University of Malaga have developed an auditory-tactile algorithm that transmits melodic information through vibrations. This algorithm uses a tactile illusion to transform monophonic music into tangible vibration-based stimulation. Experiments have shown that this "tactile stimulation" induces more positive emotions than negative ones.

[0006] Among geriatric diseases, stroke is a highly disabling condition. The number of patients in my country has exceeded 15 million. Functional recovery from stroke-induced neurological damage is largely driven by neuroplasticity, the central nervous system's ability to adapt to changes in neural structure and function in response to patient experience.

[0007] The commonly used rehabilitation principle in clinical practice is to reshape brain nerves through exercise, strengthen muscle memory, and restore patient function. Studies have shown that musical activities are related to almost every part of the brain and every neural network we know of. Sound can be used to activate the connection between the left and right sides of the brain. Whether the patient is passively or actively participating in rehabilitation treatment under music conditions, active cognitive participation is required. A 2014 study in Neuropsychology Review mentioned that "changes in the brain related to music training may have a positive effect on their cognitive effects and recovery from neural damage." Combining music-guided muscle memory generation provides an effective way to actively recover. However, this approach of combining music-guided muscle memory rehabilitation cannot be evaluated in actual application, and the existing rehabilitation training tasks are relatively fixed and the treatment effect is poor. Summary of the Invention

[0008] Based on this, it is necessary to provide a music-guided hand tactile training method to address the above technical problems.

[0009] The present application provides a music-guided hand tactile training method, including:

[0010] Detect and generate emotion recognition states;

[0011] outputting a music data sequence, and changing the output speed of the music data sequence according to an adjustment parameter;

[0012] converting the music data sequence into a finger movement sequence corresponding to the music data sequence, and stimulating corresponding fingers using the finger movement sequence;

[0013] Detecting finger movements within a stimulation completion time window, and marking a finger movement sequence according to the finger movement amplitude to obtain marked data, wherein the marked data includes completed data with a finger movement amplitude higher than expected and incomplete data with a finger movement amplitude lower than expected;

[0014] The adjustment parameters are changed using the completed data, the uncompleted data, and the emotion recognition state.

[0015] Optionally, the sequence formed by the completed data and the uncompleted data is a completion sequence.

[0016] The finger motion sequence includes several divided long period time periods, and the adjustment parameters include:

[0017] The average value of the completion sequence of the previous long period;

[0018] The evaluation value of the completion sequence fluctuation in the previous long-term period.

[0019] Optionally, the finger motion sequence includes a long period time period and short period time periods constituting the long period time period;

[0020] The completion rate of the short cycle time period is Wherein, x is the label data of the finger motion sequence, I is the number of time windows in the short period time period, for the completed data, x=1, and for the unfinished data, x=0;

[0021] The average value of the completion sequence is the φ of each short period in the long period. s The average value of the set;

[0022] The completion sequence fluctuation evaluation value is the φ of each short period in the long period s The standard deviation of the set.

[0023] Optionally, the adjustment parameters include:

[0024] The mean value of the emotion recognition state feature during the previous long period of time.

[0025] Optionally, outputting a music data sequence and changing the output speed of the music data sequence according to an adjustment parameter specifically includes:

[0026] Obtaining music data sequences corresponding to different age information stored in the cloud;

[0027] Detecting and obtaining age information of a participant, and selecting the music data sequence corresponding to the age information;

[0028] Reading an initial tempo stored locally, and outputting the music data sequence according to the initial tempo;

[0029] changing the output speed of the music data sequence according to the adjustment parameter;

[0030] Filtering each long-term time period whose completion sequence average value is greater than an expected threshold value, and obtaining the music data sequence, emotion recognition state feature average value, and music speed average value of each long-term time period that meets the screening conditions;

[0031] The emotion recognition state feature mean value that meets the expected corresponding long-period time period is screened, and the music data sequence of the corresponding long-period time period and the average music speed corresponding to the music data sequence are obtained, the age information corresponding to the music data sequence and the average music speed value are recorded and stored locally, and the average music speed value is used as the initial speed for next use.

[0032] Optionally, the output speed of the music data sequence is changed according to the adjustment parameter, using the following formula:

[0033] v t =αv t-1 +βE t-1 +γC t-1 +ηCS t-1

[0034] Where, v t is the output speed of the music data sequence in the current long period time period, v t-1 The output speed of the music data sequence in the previous long period of time, the adjustment parameters include:

[0035] E t-1 , is the mean value of the emotion recognition state characteristics in the previous long period of time;

[0036] C t-1 , is the average value of the completion sequence of the previous long period of time;

[0037] CS t-1 , is the evaluation value of the completion sequence fluctuation in the previous long-term period;

[0038] α, β, γ, and η are all preset constants.

[0039] Optionally, detect and generate emotion recognition states, including:

[0040] Performing face recognition and obtaining face recognition information;

[0041] According to the results of face recognition, identify age information;

[0042] Detect respiratory and / or electrocardiographic physiological signals to obtain physiological characteristic data;

[0043] An emotion recognition state is dynamically generated based on the face recognition information, the physiological characteristic data, and the age information.

[0044] Optionally, dynamically generating an emotion recognition state based on the face recognition information, the age information, and the physiological characteristic data specifically includes:

[0045] Obtaining a first emotion recognition state probability distribution using an emotion recognition model based on the face recognition information;

[0046] Obtaining a second emotion recognition state probability distribution based on the physiological characteristic data;

[0047] A weighted ratio of the first emotion recognition state probability distribution and the second emotion recognition state probability distribution is set according to age information to generate an emotion recognition state.

[0048] The present application also provides a music-guided hand tactile training system, comprising:

[0049] Emotion recognition module, detects and generates emotion recognition state;

[0050] a music generation module, which outputs a music data sequence and changes the output speed of the music data sequence according to an adjustment parameter;

[0051] a tactile sensation generating module, which converts the music data sequence into a finger motion sequence corresponding to the music data sequence, and stimulates corresponding fingers using the finger motion sequence;

[0052] A motion capture module detects finger movements within a stimulation completion time window and labels the finger movement sequence according to the finger movement amplitude to obtain labeled data, wherein the labeled data includes completed data with a finger movement amplitude higher than expected and incomplete data with a finger movement amplitude lower than expected;

[0053] A dynamic adjustment module changes the adjustment parameters using the completed data, the uncompleted data, and the emotion recognition state.

[0054] Optionally, the music-guided hand tactile training system includes a tactile generating glove for wear, and the tactile generating module and the motion capturing module are both built into the tactile generating glove.

[0055] The music-guided hand tactile training method of the present application has at least the following effects:

[0056] This application obtains adjustment parameters through emotion recognition status, completed data, and uncompleted data, ensuring the reliability of the adjustment parameters and thus changing the output speed of the music data sequence in real time. The completed data is tactile feedback based on muscle memory. This application combines the tactile feedback of music therapy to perform two-way adjustment on the patient's emotions, enabling real-time status evaluation and adaptive adjustment of the patient's treatment effect, thereby improving the effectiveness of rehabilitation treatment, promoting emotional stability of patients, and facilitating long-term active participation. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 This is a flowchart of a music-guided hand tactile training method in one embodiment of the present application;

[0058] Figure 2 This is a structural block diagram of a music-guided hand tactile training system in one embodiment of the present application;

[0059] Figure 3 This is a structural block diagram of the tactile generation glove in the music-guided hand tactile training system in one embodiment of the present application.

[0060] Figure 4 This is a structural block diagram of the emotion recognition module in the music-guided hand tactile training system in one embodiment of the present application. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0062] To solve the above technical problems, see Figures 1 to 4 In one embodiment of the present application, a music-guided hand tactile training system is provided, comprising:

[0063] An emotion recognition module detects and generates an emotion recognition state (corresponding to step S100);

[0064] The music generation module outputs a music data sequence and changes the output speed of the music data sequence according to the adjustment parameter (corresponding to step S200);

[0065] The tactile generation module converts the music data sequence into a finger movement sequence corresponding to the music data sequence, and stimulates the corresponding finger using the finger movement sequence (corresponding to step S300);

[0066] A motion capture module detects finger movements within a stimulation completion time window and labels the finger movement sequence according to the finger movement amplitude to obtain labeled data, wherein the labeled data includes completed data with a finger movement amplitude higher than expected and incomplete data with a finger movement amplitude lower than expected (corresponding to step S400, for example, implemented by a motion recognition and evaluation module);

[0067] The dynamic adjustment module changes the adjustment parameters using the completed data, uncompleted data, and emotion recognition status (corresponding to step S500).

[0068] Furthermore, the music-guided hand tactile training system includes a wearable tactile glove with a built-in tactile generation module and motion capture module. This differs from conventional tactile gloves used for scenario-based tasks, such as grasping or holding a cup, which lack detailed guidance.

[0069] The tactile generation module in the tactile glove of this embodiment is used to stimulate the participant. Specifically, the tactile generation module includes a stimulation device placed on each finger, which converts the manual movement sequence into stimulation data for each finger stimulation device. The tactile generation module can generate stimulation using various methods, including air pressure and electrical stimulation.

[0070] The motion capture module places measurement sensors on each finger to detect individual finger movements. Sensors available for data gloves, such as bend sensors, small linear resonant actuators (LRAs), and optical fibers, can be used to detect finger movement amplitude, such as finger bending amplitude.

[0071] The functions of each module of the music-guided hand tactile training system and the specific methods of realizing the functions are correspondingly explained in the music-guided hand tactile training method.

[0072] See Figures 1 to 4 In one embodiment of the present application, a music-guided hand tactile training method is provided, comprising the following steps:

[0073] Step S100, detecting and generating an emotion recognition state;

[0074] Step S200, outputting a music data sequence, and changing the output speed of the music data sequence according to the adjustment parameter;

[0075] Step S300, converting the music data sequence into a finger movement sequence corresponding to the music data sequence, and stimulating the corresponding fingers using the finger movement sequence;

[0076] Step S400, detecting finger movements within the stimulation completion time window, and marking the finger movement sequence according to the finger movement amplitude to obtain marked data, the marked data including completed data with finger movement amplitudes higher than expected, and incomplete data with finger movement amplitudes lower than expected;

[0077] Step S500: using the completed data, uncompleted data, and emotion recognition status to change the adjustment parameters.

[0078] In each step, the emotion recognition state can be completed using face recognition, the music data sequence can be simply understood as a music file, and the music data sequence can be converted into a finger motion sequence corresponding to the music data sequence through the "auditory tactile algorithm" provided in the background technology. It can be understood that the music data sequence and the finger motion sequence correspond to each other in time sequence.

[0079] In this embodiment, the adjustment parameters are obtained through the emotion recognition state, completed data, and uncompleted data, which ensures the reliability of the adjustment parameters. The adjustment parameters are used to change the output speed of the music data sequence in real time, thereby adjusting the difficulty of the patient's hand movements following the music. When the adjustment parameters are not in effect, the default is the output speed of the original song, that is, "1". The completed data is tactile feedback based on muscle memory. This embodiment combines the tactile feedback of music therapy to perform bidirectional regulation of the patient's emotions. It can evaluate the patient's treatment effect in real time and adjust the adaptiveness, thereby improving the effect of rehabilitation treatment, promoting the patient's emotional stability, and facilitating long-term active participation.

[0080] In step S200, the music data sequence itself is not a string of numbers, it contains interval time information. Ordinary music itself is not output uniformly, and generally different monosyllables have different durations, so one sound may be longer and another sound may be shorter. The music data sequences corresponding to different age information are set according to the age information. The music data sequences with different age information include two attributes: social preference type (for example, The East is Red and The Lone Brave belong to different age groups) and monosyllable duration (i.e., time dimension). The music data sequence will be converted into a finger motion sequence according to the auditory tactile algorithm. The output speed of the music data sequence is changed according to the adjustment parameters, for example, the time interval between the output of different sounds can be adjusted. Of course, the finger motion sequence is also adaptively adjusted at this time.

[0081] In step S200, the music data sequence is output, which specifically includes: obtaining the music data sequence corresponding to different age information stored in the cloud; detecting and obtaining the age information of the participants, and filtering and outputting the music data sequence corresponding to the age information.

[0082] Existing emotion recognition methods are either based on physiological signals or video data, but the applicability of emotion recognition varies for different groups of people.

[0083] Furthermore, in step S100, detecting and generating an emotion recognition state specifically includes: step S110, performing face recognition to obtain face recognition information, specifically by a face recognition module; step S120, identifying age information based on the face recognition result, specifically by an age recognition module; step S130, detecting respiratory and / or electrocardiographic physiological signals to obtain physiological characteristic data, specifically by a physiological signal monitoring module; step S140, dynamically generating an emotion recognition state based on the face recognition information, physiological characteristic data, and age information. In step S140, the emotion recognition module dynamically generates an emotion recognition state based on the face recognition information (video data), physiological characteristic data, and age information.

[0084] In this embodiment, multimodal sentiment analysis dynamically formed by using facial recognition information, physiological feature data, and age information can complement information between different modalities and be used for disambiguation. In combination with age information, it is integrated at the decision-making level to make sentiment analysis more accurate, more robust, and more in line with human natural expression.

[0085] Both the facial recognition module and the age recognition module can be designed using the Intel Neural Compute Stick, based on the OpenVINO AI toolkit. Physiological characteristic data includes, for example, electrocardiogram (ECG) signals and respiration signals. ECG signals can be measured using a standard design based on TI's AD8233. Respiration signals can be acquired using a data flow based on the IWR6843, an integrated single-chip frequency-modulated continuous-wave (FMCW) radar sensor. In a given close-range scenario, the point of strongest signal reflection energy is selected, representing the chest cavity of the subject. This serves as a reference for analyzing reflection phase changes to derive surface amplitude changes caused by respiration. Each 50ms frame is measured, with a set of data being measured each time. After obtaining the surface amplitude change curve, an appropriate sliding window is selected, using 512 frames of data (a sliding window of 25.6 seconds). Correlation filtering is then performed on this phase information. For respiration and heartbeat estimation, two sets of bandpass filters with different cutoff frequencies are used to filter out the respiration and heartbeat signal waveforms. Methods such as FFT or peak counting are then used to determine the subject's respiration.

[0086] Step S140 includes: step S141, obtaining the first emotion recognition state probability distribution according to the face recognition information and the emotion recognition model (specifically implemented by the video emotion recognition unit); step S142, obtaining the second emotion recognition state probability distribution according to the physiological characteristic data (specifically implemented by the physiological emotion recognition unit); step S143, setting the weighted ratio of the first emotion recognition state probability distribution and the second emotion recognition state probability distribution according to the age information (specifically implemented by the fusion unit), obtaining the overall emotion state probability distribution, and finally obtaining the emotion recognition state according to the maximum value.

[0087] The emotion recognition module includes a video emotion recognition unit, a physiological emotion recognition unit, and a fusion unit. The video emotion recognition unit obtains the emotion recognition state probability distribution A (first emotion recognition state probability distribution) based on the video data of the face recognition module and the emotion recognition model. The video emotion recognition unit can use the Intel Neural Compute Stick and be designed based on the OpenVINO artificial intelligence toolkit. The physiological emotion recognition unit obtains the emotion recognition state probability distribution B (second emotion recognition state probability distribution) based on the physiological feature data of the physiological signal monitoring module. The model can be established using the MIT emotional physiology dataset and the emotional physiology dataset of the University of Augsburg in Germany combined with deep learning methods to construct a model. The fusion unit sets different weighting ratios according to age information, fuses A and B, generates the final emotion recognition probability distribution E, and obtains the emotional state based on the maximum value of the probability distribution E.

[0088] In step S400, the sequence formed by the completed data and the uncompleted data is the completion sequence. In step S500, the dynamic adjustment module obtains the average value of the completion sequence and the average value of the completion sequence for the long-term time period, calculates and obtains the adjustment parameters, and outputs them to the music generation module. The finger movement sequence includes several divided long-term time periods. The adjustment parameters include: the average value of the completion sequence for the previous long-term time period; the fluctuation evaluation value of the completion sequence for the previous long-term time period; and the mean value of the emotion recognition state characteristics for the previous long-term time period.

[0089] Specifically, the finger motion sequence includes a long period of time T L , and the short period time periods T that make up the long period time period s The completion rate of the short cycle time period is Where x is the labeled data of the finger motion sequence, I is the number of time windows in the short cycle time period, for completed data, x = 1, for uncompleted data, x = 0; the average value of the completion sequence is φ of each short cycle time period in the long cycle time period s The average value of the set. The completion sequence fluctuation evaluation value is the φ of each short period in the long period. s The standard deviation of the set.

[0090] Furthermore, the music-guided tactile hand training method targets different tactile training tasks. For one tactile training task, the original music data sequence remains unchanged. For example, one tactile training task involves looping a certain original track for several long periods of time. This approach ensures that the adjustment parameters obtained for the previously output music data sequence do not interfere with the effect of changing the current music data sequence, thus ensuring the reliability of the music-guided tactile hand training method.

[0091] Each tactile sensation is generated as an event. In the current event, the corresponding finger receives stimulation. The motion recognition evaluation module detects the feedback of the finger after receiving the stimulation. If the detected finger movement amplitude is higher than expected, it is recorded as valid data and the completion degree of the current event is recorded as x=1, otherwise it is x=0.

[0092] In step S200, the output speed of the music data sequence is changed according to the adjustment parameter, using the following formula: t =αv t-1 +βE t-1 +γC t-1 +ηCS t-1 For long-period time periods, the current time period is t and the previous time period is t-1.

[0093] Where, v t is the output speed of the music data sequence in the current long period time period, v t-1The output speed of the music data sequence in the previous long period of time. The adjustment parameters include:

[0094] E t-1 , is the mean value of the emotion recognition state characteristics of the previous long period of time, the normal value is 0, positive emotion is 1, and negative emotion is -1;

[0095] C t-1 , is the average value of the completion sequence of the previous long period of time;

[0096] CS t-1 , is the evaluation value of the completion sequence fluctuation in the previous long-term period;

[0097] α, β, γ, and η are all preset constants. Their value range is [0-1], and their sum is 1. They are used for data normalization.

[0098] The music-guided hand tactile training method provided in various embodiments of this application is based on music-guided tactile generation and motion capture, the context of the music task based on the participant's emotional state assessment, and emotion recognition based on multimodal sentiment analysis fusion. The music-guided hand tactile training system provides a set of tactile generation gloves synchronized with music, and adjusts the music output speed based on factors such as emotion recognition, age recognition, and previous difficulty.

[0099] In one embodiment, step S500 includes:

[0100] Step S510 records the adjustment parameters corresponding to different music data sequences. Step S510 specifically includes: for a music data sequence (an original track), screening each long-term time period where the average value of the completion sequence is greater than the expected threshold, and the average value of the adjustment parameters in each long-term time period is used as the adjustment parameter for the corresponding music data sequence.

[0101] Step S520: record and store the corresponding relationship between the music data sequence and its corresponding adjustment parameters.

[0102] The results of the record storage are used to set local personalized preferences. Of course, the weighted values ​​of the personalized preference results of different users can be used as the default preferences for new users.

[0103] In contrast to the above-mentioned personalized preference setting, in one embodiment, another personalized preference setting method is provided. Step S500 includes:

[0104] For a music data sequence, filter each long-period time period whose completion sequence average value is greater than an expected threshold value, and obtain the average music speed of each long-period time period that meets the screening conditions;

[0105] The average value of the music speed is used to change the initial speed of the corresponding music data sequence.

[0106] The changed initial tempo of the music data sequence is used to perform local personalized preference settings, and this process can also be regarded as a music data update of the universal music setting database.

[0107] Specifically, use the following formula to complete, V age-new =λV age-last +κv new ; Among them, V age-new The recommended basic music speed update value under different age information of the general music setting database (the initial speed of the changed music data sequence), V age-last The existing recommended basic music speed under different age information of the general music setting database (the initial speed of the music data sequence before the change), v new The music tempo (average music tempo) of a user currently updated in the personalized music settings database. λ and κ both range from [0, 1], and their sum is 1. It can be understood that, in the factory state, the system's original standard music data sequence tempo is "1."

[0108] Specifically, the music generation module includes a general music settings database and a personalized music settings database. The general music settings database stores music clips and common basic data related to different music ages and music output speeds, and is stored in the cloud. The personalized music settings database initially contains data from the general music settings database, which is personalized and modified based on data from the music generation module and stored locally on the user's computer. The personalized music settings database periodically transmits data to the general music settings database, which is then updated based on the data from each user's personalized music settings database.

[0109] Based on the age information obtained by the age recognition module, the music generation module retrieves music data and the basic music output speed (initial speed) from the personalized music settings database. The music data includes the music data sequence and the corresponding finger movement sequence, and then sets the tactile generation glove output. Simultaneously, the music data output speed is dynamically adjusted based on the completion sequence and the emotion recognition status of the emotion recognition module, and the personalized music settings database is adjusted based on relevant data.

[0110] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0111] The technical features of the above embodiments may be combined in any manner. To simplify the description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there are no conflicts in the combination of these technical features, they should be considered to be within the scope of this specification. When technical features in different embodiments are reflected in the same figure, it can be regarded as that figure also discloses the combination examples of the various embodiments involved.

[0112] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. Music-guided hand tactile training system, characterized by: include: Emotion recognition module, detects and generates emotion recognition state; a music generation module, which outputs a music data sequence and changes the output speed of the music data sequence according to an adjustment parameter; a tactile sensation generating module, which converts the music data sequence into a finger motion sequence corresponding to the music data sequence, and stimulates corresponding fingers using the finger motion sequence; a motion capture module that detects finger movements within a stimulation completion time window and labels a finger movement sequence according to the finger movement amplitude to obtain labeled data, wherein the labeled data includes completed data with a finger movement amplitude higher than expected and incomplete data with a finger movement amplitude lower than expected. The sequence formed by the completed data and the incomplete data is a completion sequence; The dynamic adjustment module uses the completed data, the unfinished data, and the emotion recognition state to change the adjustment parameters; the finger movement sequence includes several long-period time periods, and the adjustment parameters include: the average value of the completion sequence of the previous long-period time period and the fluctuation evaluation value of the completion sequence of the previous long-period time period.

2. The music-guided hand tactile training system according to claim 1, wherein: The finger motion sequence includes a long period time period and short period time periods constituting the long period time period; The completion rate of the short cycle time period is , where x is the label data of the finger motion sequence, I is the number of time windows in the short period time period, for the completed data, x=1, and for the unfinished data, x=0; The completion sequence average is the average of each short period in the long period. The average value of the set; The completion sequence fluctuation evaluation value is the value of each short period in the long period. The standard deviation of the set.

3. The music-guided hand tactile training system according to claim 1, wherein: The music-guided hand tactile training method is targeted at different tactile training tasks, and for one of the training tasks, the original music track of the music data sequence remains unchanged; The adjustment parameter includes: the average value of the emotion recognition state characteristics in the previous long period of time.

4. The music-guided hand tactile training system according to claim 3, wherein: Outputting a music data sequence and changing the output speed of the music data sequence according to an adjustment parameter specifically include: Obtaining music data sequences corresponding to different age information stored in the cloud; Detecting and obtaining age information of a participant, and selecting the music data sequence corresponding to the age information; Reading an initial tempo stored locally, and outputting the music data sequence according to the initial tempo; changing the output speed of the music data sequence according to the adjustment parameter; Filtering each long-term time period whose completion sequence average value is greater than an expected threshold value, and obtaining the music data sequence, emotion recognition state feature average value, and music speed average value of each long-term time period that meets the screening conditions; The emotion recognition state feature mean value that meets the expected corresponding long-period time period is screened, and the music data sequence of the corresponding long-period time period and the average music speed corresponding to the music data sequence are obtained, the age information corresponding to the music data sequence and the average music speed value are recorded and stored locally, and the average music speed value is used as the initial speed for next use.

5. The music-guided hand tactile training system according to claim 3, wherein: The output speed of the music data sequence is changed according to the adjustment parameter, using the following formula: Where, is the output speed of the music data sequence in the current long period time period, The output speed of the music data sequence in the previous long period of time, the adjustment parameters include: , is the mean value of the emotion recognition state characteristics in the previous long period of time; , is the average value of the completion sequence of the previous long period of time; , is the evaluation value of the completion sequence fluctuation in the previous long-term period; 、 、 、 , are preset constants.

6. The music-guided hand tactile training system according to claim 1, wherein: Detect and generate emotion recognition states, including: Performing face recognition and obtaining face recognition information; According to the results of face recognition, identify age information; Detect respiratory and / or electrocardiographic physiological signals to obtain physiological characteristic data; An emotion recognition state is dynamically generated based on the face recognition information, the physiological characteristic data, and the age information.

7. The music-guided hand tactile training system according to claim 6, wherein: Dynamically generating an emotion recognition state based on the face recognition information, the age information, and the physiological characteristic data, specifically including: Obtaining a first emotion recognition state probability distribution using an emotion recognition model based on the face recognition information; Obtaining a second emotion recognition state probability distribution based on the physiological characteristic data; A weighted ratio of the first emotion recognition state probability distribution and the second emotion recognition state probability distribution is set according to age information to generate an emotion recognition state.

8. The music-guided hand tactile training system according to claim 1, wherein: The music-guided hand tactile training system includes a tactile generating glove for wear, and the tactile generating module and the motion capturing module are both built into the tactile generating glove.

Images