Emotional interaction regulation and control strategy generation method and device for rehabilitation training

Abstract

The invention provides an emotional interaction regulation and control strategy generation method and device for rehabilitation training, and the method comprises the steps: obtaining the psychological and physical states of a user according to the collected physiological and behavior signals of the user in the rehabilitation training process; and for the psychological and physical states of the user, simulating professional knowledge and thinking modes of a rehabilitation physician based on a large language model to understand the current state of the user and fuse reasoning decision, and generating a multi-dimensional emotion interaction regulation strategy which comprises training contents of a rehabilitation training device. And auditory and visual interaction contents with interpersonal interaction factors are fused between the rehabilitation training device and the user. By simulating the positive emotion effect in the interpersonal interaction process, the interaction regulation and control strategy which can be executed by external interaction equipment such as visual and auditory interaction equipment and rehabilitation training equipment is output, immersive rehabilitation experience is provided for the user, the positive emotion and training motivation of the user are motivated, and therefore the rehabilitation effect is improved, and the user experience is improved. The recovery of the motion function of the user is promoted.

Description

Technical Field

[0001] The present invention belongs to the technical field of elderly care, rehabilitation and human-computer interaction, and in particular relates to a method and device for generating an emotion interaction regulation strategy for rehabilitation training. Background Art

[0002] With the development of global deep aging, the number of elderly people with disability, dementia, and cognitive impairment continues to increase. Most of them have limb movement disorders and suffer from both physical and psychological torture. Medical theory and clinical medicine have proved that correct and scientific rehabilitation training plays a very important role in the recovery and improvement of motor function. In order to solve the problems of lack of professional caregivers and expensive medical costs, safe, quantitative, effective and repetitive rehabilitation training devices have emerged. However, the rehabilitation process brought by purely mechanical equipment is often boring and monotonous, and the existing rehabilitation equipment is far from replacing the emotional support provided by doctors during the patient's rehabilitation process, which makes the patient's emotional state negatively affected, greatly affecting their rehabilitation efficiency.

[0003] An excellent rehabilitation physician can not only flexibly adjust the training tasks according to the patient's current condition to stimulate their confidence and motivation for rehabilitation, but also provide them with great emotional help through appropriate communication. A good doctor-patient relationship has been proven to play a vital role in improving medical outcomes, including motor learning performance. Therefore, solving the problem that existing rehabilitation devices still cannot replace rehabilitation physicians, giving robots emotional interaction capabilities comparable to rehabilitation physicians, introducing the positive effects brought about by interpersonal emotional communication, and creating an immersive and positive rehabilitation environment to enhance the rehabilitation effect are important development directions of sports rehabilitation technology. The key technology to solve this problem is to introduce autonomous and real-time emotional interaction regulation strategies in the rehabilitation training process led by automated rehabilitation devices, so as to provide timely and effective emotional support based on the patient's training performance and improve their rehabilitation effect. However, this key technology has not yet received enough attention, and there is no existing technology yet. Summary of the invention

[0004] The present disclosure aims to solve one of the technical problems existing in the existing related technologies at least to a certain extent.

[0005] To this end, the disclosed embodiments propose a method and device for generating an emotion interaction regulation strategy for rehabilitation training, which utilizes deep learning and a generative large language model to simulate the positive emotion effects in the process of interpersonal interaction, incorporates the functions of interpersonal emotion interaction and emotion regulation into the design of the rehabilitation system, and outputs interaction regulation strategies that can be executed by external interaction devices such as visual and auditory interaction devices and rehabilitation training devices, thereby providing users with an immersive rehabilitation experience, stimulating users' positive emotions and training motivation, thereby improving the rehabilitation effect and promoting the recovery of users' motor functions.

[0006] In order to achieve the above objectives, the present disclosure adopts the following technical solutions:

[0007] The first aspect of the present disclosure provides a method for generating an emotion interaction regulation strategy for rehabilitation training, comprising:

[0008] Step S100, obtaining the user's psychological and physical state according to the collected physiological and behavioral signals of the user during the rehabilitation training process;

[0009] Step S200: Based on the large language model, the professional knowledge and thinking mode of the rehabilitation physician are simulated to understand the user's current state and make integrated reasoning decisions, and generate a multi-dimensional emotional interaction regulation strategy, including the training content of the rehabilitation training device, and the auditory and visual interaction content between the rehabilitation training device and the user that integrates interpersonal interaction factors.

[0010] In some embodiments, in the emotion interaction regulation strategy generation method, in step S100, the process of obtaining the user's psychological state includes: preprocessing and preliminary feature extraction of each type of acquired signal; then using a deep learning-based classification model to perform deep feature extraction on the extracted various types of preliminary features to obtain the user's psychological state.

[0011] In some embodiments, the emotional interaction regulation strategy generation method divides the user's psychological state into an emotional state and a mental load level, divides the emotional state into three levels: positive, neutral, and negative, and divides the mental load level into three levels: high, medium, and low;

[0012] The deep learning-based classification model is used to perform deep feature extraction on various extracted preliminary features to obtain the user's psychological state, specifically including:

[0013] Each type of extracted preliminary features is taken as a modality, and a classification model based on deep learning is used to perform single-modal feature fusion and classification to obtain the preliminary psychological state classification results of the user; wherein, each modality uses a corresponding classification model, and during the training process of each classification model, a labeled data set is used for supervised learning, and the labels are evaluated based on the subjective emotion scale and the perceived stress scale. Through the scores of the subjects, the scores are divided into high, medium, and low training labels according to the scale standards to classify emotions and mental loads;

[0014] Based on the rules, all the preliminary psychological state classification results of single modalities are subjected to multimodal decision fusion, wherein the rule is to dynamically adjust the weight of each modality in the decision fusion according to its individual classification performance, and the modality with better classification performance is given a higher weight. The weighted result is obtained by linearly weighted summing the classification labels of each modality, and the weighted result is normalized to obtain the user's final psychological state classification result as the discrimination result of the user's psychological state.

[0015] In some embodiments, the method for generating an emotion interaction regulation strategy, in step S100, divides the user's physical state into peripheral fatigue degree and rehabilitation task performance;

[0016] The peripheral fatigue degree is used to directly reflect the muscle state of the user, and is divided into three levels: high, medium, and low. The peripheral fatigue information is obtained according to the user's electromyographic signal, specifically including: performing a preprocessing operation on the electromyographic signal to obtain the peak electromyographic amplitude A of the current period, and comparing the peak electromyographic amplitude A with the peak electromyographic amplitude A within the first 10 seconds of training. max When A / A max When it is between 80% and 100%, it is considered as low fatigue state. max Between 60% and 80%, it is considered a moderate fatigue state. max When it is 60% or below, it is considered as a high fatigue state;

[0017] The performance of the rehabilitation task is related to the type and content of the rehabilitation training task. The evaluation indicators of the performance of the rehabilitation task include movement speed, movement smoothness, movement accuracy and evaluation indicators for the coordinated movement of bilateral limbs.

[0018] In some embodiments, the method for generating an emotion interaction regulation strategy further comprises, in step S100: integrating the user's physical state and psychological state, and outputting them in a unified natural language format:

[0019] "--Mental State--

[0020] Emotional state: positive / neutral / negative;

[0021] Mental workload: high / medium / low;

[0022] --Physical Condition--

[0023] Peripheral fatigue: high / medium / low;

[0024] Task performance: %a"

[0025] Among them, "%a" consists of specific text and a numerical value, indicating the name of the quantitative evaluation index of the rehabilitation training device and its corresponding numerical value.

[0026] In some embodiments, in the method for generating an emotion interaction regulation strategy, in step S200, the large language model is a pre-trained first large language model, and the pre-training process of the first large language model includes:

[0027] Step S210, thinking chain training: performing the first fine-tuning training on the thinking chain of the first language model, by inputting typical cases of sports rehabilitation and theories of sports rehabilitation and disease psychology, training the first language model to gradually think and reason about the content of rehabilitation training and the ability to communicate content and attitude, wherein the rehabilitation training content includes training duration, training mode and task difficulty;

[0028] Step S220, interactive text training: collect aging-friendly corpus and sports training motivational corpus to perform secondary fine-tuning training on the text content output by the first language model after the first fine-tuning, so that the interactive text output by the model is more suitable for the rehabilitation training scenario and meets user expectations.

[0029] In some embodiments, the method for generating an emotion interaction regulation strategy, step S210 specifically includes:

[0030] Step S211: Collection of existing cases:

[0031] Collect existing decision-making cases and decision-making ideas of licensed professional rehabilitation physicians when they provide rehabilitation training to users. An existing case i should at least record: (1) the user's condition s i , including injury time, lesion area and movement assessment; (2) initial artificial rehabilitation training program, which is equivalent to the initial rehabilitation training program adapted to the rehabilitation training device p i and the initial rehabilitation training program p i Normalized representation as a text sequence: "Training content: passive / assisted / active, resistance level, training speed, trajectory difficulty"; (3) The jth real-time adjustment strategy a for training content and communication content during rehabilitation training ij and reasons for adjustment ij , where strategy a will be adjusted ij The text normalization of the training content is: "Training content adjustment: passive / assisted / active, resistance level, training speed, trajectory difficulty; communication content adjustment: communication attitude, communication text", and the communication attitude has two labels: "comfort" or "encouragement"; the reason for adjustment is: ij This includes changes in the user's mental state, physical state, and / or professional theoretical knowledge that the physician refers to when making adjustments;

[0032] Step S212: Use the second language model to retrieve professional knowledge and expand the real-time adjustment reasons for the training content and communication content in the existing cases through reasoning analysis.ij :

[0033] By designing prompt words, the second largest language model is prompted to retrieve professional rehabilitation theory and disease psychology theory. Based on the zero-sample thinking chain training method, instructions for the second largest language model to think step by step are added to the prompt words, so that the second largest language model can combine the retrieved knowledge to adjust the strategy a in the existing case i. ij Conduct step-by-step analytical reasoning to supplement and improve the reasons for adjustment ij , and the single adjustment strategy a ij Corresponding adjustment reason r ij The number of characters in the inference text is limited to not exceed the first maximum length Lr 1 ;

[0034] Step S213: Data standardization:

[0035] All text s of case i i 、p i 、a ij 、r ij Generate text combination w according to the specified format i , and use all case text combinations to generate the first text sequence W = {w 1 ,w 2 ,…,w i ,…,w N};

[0036] Step S214: first fine-tuning training:

[0037] The standardized first text sequence W is organized into a first fine-tuning dataset, and the first language model is trained using the adapter fine-tuning method and the Hugging Face Transformers framework; wherein the first cross entropy loss function L is used. LOSS1 As the objective function for the adapter parameter θ 1 To optimize:

[0038]

[0039] In the formula, |r i | represents the total number of training adjustment strategy-adjustment reason groups for case i; P(p i |p i ,s i θ 1 ) represents the first language model input user condition s after the first training i The output is then used to obtain the initial rehabilitation training program p i The probability of quantifying the ability to infer the initial training plan from the user's condition; P(a i,j |pi ,r i,≤j ,s i ,a i,<j θ 1 ) is used to quantify the first language model after the first training. Based on the current user's condition, the initial rehabilitation training plan, the rehabilitation training adjustment strategy and adjustment reason before the jth adjustment, and the adjustment reason for the jth adjustment, the jth adjustment strategy a is inferred. i,j The probability of i,≤j represents the adjustment reason for the jth and previous rehabilitation training, a i,<j Represents the rehabilitation training adjustment strategy before the jth adjustment.

[0040] In some embodiments, the method for generating an emotion interaction regulation strategy, step S220 specifically includes:

[0041] Step S221: Corpus collection:

[0042] Collect aging-friendly corpus and rehabilitation exercise training motivational corpus from the Internet through web crawlers;

[0043] Step S222: Data cleaning and preprocessing:

[0044] The collected corpus is cleaned, including removing duplicate texts, correcting punctuation errors and grammatical confusion, and correcting typos and language irregularities in the corpus; the cleaned corpus is divided into different categories according to semantic functions, including motivational sentences, comforting sentences, task description sentences, and daily communication sentences; the maximum number of characters in a single corpus segment is limited to no more than the second maximum length Lr 2 ; Arrange all the obtained corpus fragments into a second text sequence V = {v 1 ,v 2 ,…,v l ,…,v M},v l Represents a single corpus fragment;

[0045] Step S223: Second fine-tuning training

[0046] The standardized second text sequence V is organized into the second fine-tuning dataset, and the first language model after the first fine-tuning is trained using the adapter fine-tuning method and based on the Hugging Face Transformers framework; wherein the second cross entropy loss function L is used LOSS2 As the objective function, fine-tune the model parameters θ 2 :

[0047]

[0048] In the formula, P(vl θ 2 ) represents the first language model prediction corpus segment v after secondary fine-tuning l probability.

[0049] In some embodiments, the method for generating an emotion interaction regulation strategy, step S200 further includes designing prompt words to enable the pre-trained first language model to generate a standardized output, and the output content includes:

[0050] "--Training content--

[0051] Training duration: %c minutes;

[0052] Training mode: active / assisted / passive;

[0053] Training impedance: %dN;

[0054] --Virtual human interaction content--

[0055] Interaction text: %e;

[0056] Interaction attitude: comfort / motivation.

[0057] --Reason for adjustment--

[0058] %f."

[0059] Among them, "%c" and "%d" are specific values, and "%e" and "%f" and "%g" are specific text contents.

[0060] A second aspect of the present disclosure provides a device for generating an emotion interaction regulation strategy for rehabilitation training, characterized by comprising:

[0061] A user state analysis module is configured to obtain the user's psychological and physical state based on the collected physiological and behavioral signals of the user during the rehabilitation training process;

[0062] The emotion interaction regulation strategy generation module is configured to understand the user's current state and make integrated reasoning decisions based on the user's psychological and physical state by simulating the professional knowledge and thinking mode of rehabilitation physicians based on a large language model, and generate a multi-dimensional emotion interaction regulation strategy, including the training content of the rehabilitation training device, and the auditory and visual interaction content between the rehabilitation training device and the user that integrates interpersonal interaction factors. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 It is a flow chart of a method for generating an emotion interaction regulation strategy for rehabilitation training provided by an embodiment of the present disclosure;

[0064] Figure 2 yes Figure 1The schematic diagram of the emotional interaction regulation strategy generation method for classifying and rating the psychological and physical states of users;

[0065] Figure 3 yes Figure 1 The schematic diagram of the specific process of analyzing the user's psychological and physical state by the emotion interaction regulation strategy generation method shown;

[0066] Figure 4 yes Figure 1 Schematic diagram of the specific process of generating a multi-dimensional emotion interaction regulation strategy. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0068] On the contrary, the present application covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present application as defined by the claims. Further, in order to make the public have a better understanding of the present application, some specific details are described in detail in the detailed description of the present invention below. For those skilled in the art, the present application can be fully understood without the description of these details.

[0069] like Figure 1 As shown, a method for generating an emotion interaction regulation strategy for rehabilitation training according to an embodiment of the present disclosure includes the following steps:

[0070] Step S100, user status analysis: obtaining the user's psychological and physical status according to the collected physiological and behavioral signals of the user, i.e., the patient, during the rehabilitation training process;

[0071] Step S200, generating an emotion interaction regulation strategy: based on the user's psychological and physical state obtained in step S100, the professional knowledge and thinking mode of a rehabilitation physician are simulated by a large language model to understand the user's current state and make integrated reasoning decisions, and generate a multi-dimensional emotion interaction regulation strategy, including the training content of the rehabilitation training device, and the auditory and visual interaction content between the rehabilitation training device and the user that integrates interpersonal interaction factors.

[0072] In some embodiments, in order to provide users with an immersive rehabilitation experience, stimulate users' positive emotions and training motivation, and thus improve rehabilitation effects, it is necessary to accurately identify the user's current physiological and psychological conditions before generating a multi-dimensional emotional interaction regulation strategy. Figure 2, the user's psychological state can be further subdivided into emotional state and mental load degree, and divided into three levels, namely positive, neutral, and negative emotions, and high, medium, and low mental load. In order to identify the user's emotional state and mental load degree label, the disclosed embodiment analyzes the user's current psychological and physical state based on the user's physiological and behavioral signals obtained during the rehabilitation training process, including but not limited to EEG, ECG, skin electricity, facial expression, voice, eye tracking, sports task performance and other data.

[0073] In some embodiments, see Figure 3 In step S100, the step of determining the user's psychological state in real time based on the acquired physiological and behavioral signals of the user includes:

[0074] Step S110, preprocessing and preliminary feature extraction are performed on each type of acquired signal;

[0075] Step S120: Use a deep learning-based classification model to perform deep feature extraction on the various extracted preliminary features to obtain the user's psychological state.

[0076] Furthermore, step S110 specifically includes the following steps:

[0077] De-noising the EEG signal data, and using independent principal component analysis (ICA) to remove eye movement and heartbeat artifacts, and using Fourier transform or wavelet transform to extract frequency domain features such as the power spectral density of the alpha, beta, theta and delta frequency bands in the EEG signal;

[0078] For ECG signal data, it is preprocessed by removing baseline drift, denoising and detecting R waves, and then extracting time domain features such as heart rate variability and RR interval, and using Fourier transform to extract low-frequency and high-frequency components of the signal;

[0079] For the skin electrical signal data, firstly, the preprocessing operation of denoising and sliding average artifact removal is performed, and then the amplitude characteristics of the skin electrical response in the target time period are extracted;

[0080] For the facial expression signal data, each frame of the collected image is pre-processed by denoising and normalization, and the key points of the user's facial features are obtained by combining the opencv facial feature detection method, and then the facial action unit features are extracted based on the Facial Action Coding System (FACS);

[0081] For speech signal data, denoising preprocessing is first performed, and then the fundamental frequency, pitch characteristics and Mel frequency cepstrum coefficients are extracted based on Fourier transform to segment the speech signal into short-time frames, calculate the number of speech frames per unit time, obtain the speech speed characteristics, calculate the energy of each frame of speech signal, and estimate its volume characteristics;

[0082] For the eye tracking signal data, background denoising preprocessing is first performed, and then the movement information of the eyeball is obtained, the position coordinates of the eye at each moment are output, and the trajectory sequence formed by the position coordinates in the time step is marked. The Kalman filter method can be used to perform smooth interpolation processing on the trajectory sequence to obtain the final eye movement trajectory characteristics.

[0083] Furthermore, in step S120, when deep feature extraction is performed on various preliminary features obtained in step S110, each type of preliminary feature is taken as a modality, and a classification model based on deep learning is first used to perform single-modal feature fusion and classification to obtain the user's preliminary psychological state label, that is, the emotional state level (including three levels of positive, neutral, and negative) and the mental load level (including three levels of high, medium, and low); then, all single-modal classification results are decision-fused based on rules to obtain the user's final psychological state label. This process can be achieved by selecting a classification model based on deep learning such as convolutional neural networks to fuse and classify various preliminary features. In order to avoid redundancy and model overfitting caused by the high dimensionality of multimodal features, a hybrid multimodal fusion strategy can be adopted to combine feature fusion and decision fusion to classify psychological states. The specific steps are as follows:

[0084] Single-modal feature fusion stage: n preliminary features f of k-modal signals k1 ,f k2 ,…,f kn Perform fusion and splicing to form the feature vector f corresponding to mode k k , train independent classification models for signals of each modality, and the classification models corresponding to different modalities can be homogeneous (i.e., the same model structure) or heterogeneous (different model structures). In one embodiment of the present application, the convolutional neural network CNN, the recurrent neural network RNN, and the long short-term memory model LSTM are trained according to the characteristics of the modal features to achieve classification, and the classification label y of the single modality k is obtained. k, the classification label corresponds to the user's preliminary psychological state label, that is, the emotional state level (divided into three types: positive, neutral, and negative), and the mental load level (divided into three types: high, medium, and low). During the training process of each classification model, a labeled data set is used for supervised learning, and the labels are assessed based on the subjective emotion scale (such as PANAS) and the perceived stress scale. Through the subjective scoring of the subjects, the scores are divided into high, medium, and low training labels according to the scale standards for emotion and mental load classification; during the training process, the cross entropy loss function can be minimized to optimize the classification accuracy of the classification model, and the K-fold cross classification method can be used to verify the classification effect of the pre-trained classification model.

[0085] Multimodal decision fusion stage: Based on the classification results of each modality obtained in the single-modal feature fusion stage, rules are formulated for decision fusion. Specifically, the classification effect of each modality can be evaluated based on the aforementioned K-fold cross-validation method. The indicators used are such as accuracy, precision, recall, and F1 score. The weight of each modality in decision fusion is dynamically adjusted according to its individual classification performance. k , giving higher weights to the modalities with better classification performance. Then the classification labels y of each modality k Perform linear weighted summation to obtain the weighted result Σr k y k , the weighted result is passed through a fully connected layer with softmax as the operation function, and the final classification label of the psychological state is output as the discrimination result of the user's psychological state.

[0086] In some embodiments, see Figure 2 In step S100, the step of analyzing the user's physical state includes:

[0087] Step S130: Classify the user's physical state into peripheral fatigue degree and rehabilitation task performance according to the acquired physiological signals (mainly electromyographic signals) and behavioral signals, where:

[0088] The degree of peripheral fatigue can directly reflect the muscle state of the user, which is also divided into three levels: high, medium and low. The electromyographic signal can be used to obtain peripheral fatigue information. The specific steps are: the obtained electromyographic signal data is preprocessed by removing baseline drift, denoising, and sliding average to obtain the peak electromyographic amplitude A of the current period, and the peak electromyographic amplitude A is compared with the peak electromyographic amplitude A within the first 10 seconds of training. max When A / A max When it is between 80% and 100%, it is considered as low fatigue state. max Between 60% and 80%, it is considered a moderate fatigue state. max When it is 60% or below, it is judged as a high fatigue state.

[0089] The performance of rehabilitation tasks is directly related to the type and content of rehabilitation training tasks. Some typical rehabilitation training task performance indicators include movement speed, movement smoothness, movement accuracy, and evaluation indicators for coordinated movement of bilateral limbs. The performance of rehabilitation tasks can be obtained by the encoder and force sensor built into the rehabilitation training device. Among them, the movement speed is characterized by the movement speed of its own mechanism collected by the rehabilitation training device (for example, for lower limb rehabilitation training tasks, it is specifically the circumferential speed of the rehabilitation training mechanism); the movement smoothness is characterized by the ratio of the average speed and peak speed of the current training stage collected by the rehabilitation training device; the movement accuracy is characterized by the degree of consistency between the movement trajectory of its own mechanism collected by the rehabilitation training device and the target trajectory path; the evaluation indicator of coordinated movement of bilateral limbs can be specifically the movement symmetry, which is characterized by the average positive pressure ratio of the left and right limbs applied to the end of the rehabilitation training device in the current training stage collected by the rehabilitation training device.

[0090] In some embodiments, step S100 further includes:

[0091] Step 140: Integrate the identified physical and psychological states of the user, and output the identification results in natural language by writing Python code. The output natural language is written in the following unified format:

[0092] "--Mental State--

[0093] Emotional state: positive / neutral / negative;

[0094] Mental workload: high / medium / low;

[0095] --Physical Condition--

[0096] Peripheral fatigue: high / medium / low;

[0097] Task performance: %a."

[0098] Among them, "%a" consists of specific text and a numerical value, indicating the name of the quantitative evaluation index of the rehabilitation training device and its corresponding numerical value.

[0099] In some embodiments, see Figure 4 Step S200 takes the generative large language model as the core, and the output multi-dimensional emotional interaction regulation strategy includes voice and visual interaction content with positive emotions such as comfort and encouragement, as well as training content for rehabilitation training devices. The flexible interactive control method of the multi-dimensional emotional interaction regulation strategy is significantly better than the preset control program used in previous human-computer interaction, and shows great potential in simulating the human thinking decision-making process and real interpersonal emotional interaction. Among them:

[0100] The voice interaction content with the user may include language text and voice audio generated based on the language text. The language text content is fully integrated with the user's actual training performance, and can reinforce the user's correct or good behavior and correct the wrong or poor behavior through positive language and text with a motivating, encouraging, and comforting style;

[0101] Visual interaction content can rely on existing visual interaction devices to show users virtual rehabilitation doctors or other virtual characters. The virtual characters can communicate with users through facial expressions, gestures, and body postures, and establish positive "interpersonal" relationships with users through voice audio, thereby stimulating users' positive emotions during rehabilitation training.

[0102] The training content of the rehabilitation training device may include information such as training duration, training mode, and task difficulty. The characteristics include: the training content should be reasonably set to maintain a certain degree of challenge, but can give users a sense of success in achieving their goals, thereby stimulating users' confidence and motivation in training.

[0103] Furthermore, in order to improve the performance and adaptability of the generative large language model in rehabilitation training scenarios and make the multi-dimensional emotional interaction regulation strategy it outputs better meet the actual needs of the user group, it is necessary to pre-train the large language model.

[0104] In some embodiments, the generative large language model adopts the first large language model, which is a lightweight large language model, such as the Llama3-8B large language model. Since the parameters of the lightweight large language model are smaller in magnitude, the computing power requirements for its pre-training process are easier to meet, and the real-time nature of the data generated by the lightweight large language model is higher, meeting the real-time requirements for interaction with rehabilitation training users. The steps of pre-training the first large language model include:

[0105] Step S210, chain-of-thought training, i.e., performing the first fine-tuning training on the chain-of-thought (CoT) of the first language model, by inputting typical cases of sports rehabilitation and theories of sports rehabilitation and disease psychology, to train the first language model to gradually think and reason about the content of rehabilitation training and the ability to communicate content and attitude;

[0106] Step S220, interactive text training, collects aging-friendly corpus and sports training motivational corpus to perform secondary fine-tuning training on the interactive text content output by the first language model, so that the interactive text output by the model is more suitable for the rehabilitation training scenario and meets user expectations.

[0107] Furthermore, the specific steps of step S210, thinking chain training, include:

[0108] Step S211: Collection of existing cases.

[0109] Collect existing decision-making cases and decision-making ideas of licensed professional rehabilitation physicians when they provide rehabilitation training to users. Optionally, an existing case i should at least record: (1) the user's condition s i , such as injury time, lesion area, movement assessment, etc.; (2) initial artificial rehabilitation training program, such as exercise training method, training speed, training duration, etc. The initial artificial exercise rehabilitation program needs to be approximately equivalent to the initial rehabilitation training program p adapted to the rehabilitation training device i , and combined with the characteristics of the rehabilitation training device i Normalized representation is a text sequence, for example: "Training content: passive / assisted / active, resistance level, training speed, trajectory difficulty". The equivalent process can refer to the existing experience in the rehabilitation field and the advice of professional rehabilitation physicians; (3) The jth real-time adjustment strategy for training content and communication content during rehabilitation training a ij and reasons for adjustment ij , where strategy a will be adjusted ij The text normalization of the training content is: "Training content adjustment: passive / assisted / active, resistance level, training speed, trajectory difficulty; communication content adjustment: communication attitude, communication text", further, the communication attitude can be "comfort" or "encouragement" two labels; adjustment reason r ij It may include changes in the user's psychological state, physical state, and further include professional theoretical knowledge that the physician refers to when making adjustments.

[0110] Step S212: Use the second language model to retrieve professional knowledge and expand the real-time adjustment reasons for the training content and communication content in the existing cases through reasoning analysis. ij .

[0111] By designing prompts, the second language model is prompted to search a wide range of professional rehabilitation theories and disease psychology theories. Based on the zero-sample thinking chain training method, instructions are added to the prompt to make the second language model think step by step, so that the second language model can combine the retrieved knowledge to adjust the strategy a in the existing case i. ij The second language model is based on the retrieved knowledge and the original professional rehabilitation physician’s experience and knowledge to adjust the strategy a ij Conduct step-by-step analytical reasoning to supplement and improve the reasons for adjustment ij To improve training efficiency, compared with the single adjustment strategy a ij Corresponding adjustment reason r ij The number of characters in the inference text is limited to not exceed the first maximum length Lr 1= 256. Optionally, the second largest language model used in this embodiment is ChatGPT-4o, so as to fully combine external knowledge and existing case information for efficient reasoning and supplementation, so as to expand the sample size required for offline pre-training of the first largest language model.

[0112] Step S213: data standardization.

[0113] Case i all text s i 、p i 、a ij 、r ij Generate text combination w according to the specified format i , and use all case text combinations to generate the first text sequence W = {w 1 ,w 2 ,…,w i ,…,w N}, since the first text sequence can form a causal chain relationship, the w in the first text sequence W i It can be expressed in the format (s i -->p i ,j:r ij -->a ij ).

[0114] Step S214: first model fine-tuning training.

[0115] The standardized first text sequence W is organized into the first fine-tuning data set, and the first fine-tuning training is performed on the fully open source Llama3-8B large language model. Specifically, the first fine-tuning process adopts the adapter fine-tuning method, that is, a lightweight adapter module is introduced in each layer of the pre-trained first large language model, and the weight parameters of these adapter modules are adjusted through training, while keeping the original parameters of the first large language model unchanged, thereby achieving efficient fine-tuning of the first large language model. Adapter Tuning not only greatly reduces the amount of parameters and resource consumption required in the training process, but also has good task adaptability and scalability, and is suitable for fine-tuning requirements of multi-tasks or specific scenarios. The fine-tuning process of the Llama3-8B large language model is implemented based on the framework of Hugging Face Transformers. HuggingFace provides a convenient TrainerAPI and efficient fine-tuning tools that can quickly integrate Adapter Tuning and optimize the training process. Further, Hugging Face's Accelerate library can be combined to achieve distributed training and automatic mixed precision training, thereby improving fine-tuning efficiency and large language model performance. In a specific embodiment, the fine-tuning process sets the learning rate to 3e-4, the number of training epochs to 20, and uses the first cross entropy loss function L LOSS1 The parameter θ1 of the adapter module is optimized as the objective function, as follows:

[0116]

[0117] In the formula, |r i | represents the total number of training adjustment strategy-adjustment reason groups for case i. Since different cases may have different numbers of training adjustment strategies-adjustment reasons, in order to ensure that the weights of the loss function are constant when different cases are trained on the thinking chain and are not affected by the number of adjustment strategies-adjustment reasons, |r is used before the second term of the formula. i |Perform a normalization; P(p i |p i ,s i θ 1 ) indicates the first language model after training input user's condition s i The output is then used to obtain the initial rehabilitation training program p i The probability of quantifying the ability to infer the initial training plan from the user's condition; P(a i，j |p i , r i，≤j ,s i , a i，＜j θ 1) is used to quantify the first language model after training. Based on the current user's condition, the initial rehabilitation training plan, the rehabilitation training adjustment strategy and adjustment reason before the jth adjustment, and the adjustment reason for the jth adjustment, the jth adjustment strategy a is inferred. i，j The probability of i，≤j represents the adjustment reason for the jth and previous rehabilitation training, a i，＜j Represents the rehabilitation training adjustment strategy before the first adjustment.

[0118] Furthermore, the specific steps of step S220, interactive text training, include:

[0119] Step S221: Corpus collection.

[0120] Aged-friendly corpus and rehabilitation exercise training motivational corpus are collected from the Internet by means of web crawlers. Optionally, the aged-friendly corpus involves the language expressions commonly used by the elderly in daily life, health management related to the elderly, psychological intervention, social support and other contents; its sources include: online articles, forums and comments related to elderly health and psychological support, case corpus of communication with the elderly in social support and companionship services, and corpus describing the elderly in rehabilitation medical institutions and professional literature. The rehabilitation exercise training motivational corpus is mainly motivational corpus to enhance the user's confidence and motivation in rehabilitation training; its sources include: the compilation of motivational sentences in professional rehabilitation training documents; the real dialogue records between physicians and users in excellent rehabilitation cases; and literature and corpus related to motivation in sports psychology research.

[0121] Step S222: data cleaning and preprocessing.

[0122] The collected corpus is cleaned, including removing duplicate text, correcting punctuation errors and grammatical confusion, and correcting typos and language irregularities in the corpus through automated tools. In some embodiments, a python library named pycorrector can be used to implement Chinese text error correction. After the data cleaning is completed, the corpus is divided into different categories according to semantic functions, such as motivational sentences, comforting sentences, task description sentences, daily communication sentences, etc., so that the subsequent first language model can output appropriate text to the user in the rehabilitation training scenario. To ensure the uniformity and efficient processing of the corpus, the maximum number of characters for a single corpus segment is limited to Lr 2 =128 to avoid the interference and influence of the super-long corpus on the rehabilitation training process. Finally, the obtained corpus is formatted into the second text sequence V = {v 1 , v 2 , ..., v l , ..., v M}, where v l Represents a single corpus fragment, and there is no causal relationship between the corpus fragments.

[0123] Step S223: secondary model fine-tuning training.

[0124] The standardized second text sequence V is organized into a second fine-tuning dataset, and the fine-tuned Llama3-8B large language model obtained in step S214 is subjected to secondary fine-tuning training. For the specific fine-tuning training process, see step S214. The learning rate used in the secondary fine-tuning process can be 2e-5, the epoch is set to 20, and the second cross entropy loss function L is introduced. LOSS2 As the objective function of training, fine-tune the model parameters θ 2 , as shown below:

[0125]

[0126] In the formula, P(v l θ 2 ) represents the first language model prediction corpus segment v after secondary fine-tuning l probability.

[0127] After the second fine-tuning, the first language model has a significant performance improvement in the formulation of human-computer interaction strategies in rehabilitation training scenarios. It can fully analyze the user's multi-dimensional psychological and physical conditions and make interactive decisions in the way of thinking of an excellent rehabilitation physician, including appropriately adjusting the content and difficulty level of the next stage of rehabilitation training tasks to stimulate the user's confidence and motivation to actively participate in rehabilitation training, and timely generating some comforting or encouraging texts to directly provide emotional support to users. By simulating the real doctor-user and family-user interpersonal interaction process in the human-computer interaction process, it fully integrates and utilizes the positive effects of interpersonal emotional interaction in sports learning, improves the user's mental health level, and amplifies the benefits of rehabilitation training tasks on the user's physical condition.

[0128] In some embodiments, step S200 further includes:

[0129] Step S230: Design prompt so that the fine-tuned first language model can generate standardized output after receiving standardized input. The specific output content includes:

[0130] "--Training content--

[0131] Training duration: %c minutes;

[0132] Training mode: active / assisted / passive;

[0133] Training impedance: %dN.

[0134] --Virtual human interaction content--

[0135] Interaction text: %e;

[0136] Interaction attitude: comfort / motivation.

[0137] --Reason for adjustment--

[0138] %f."

[0139] In the above text information, "%c" and "%d" are specific values, and "%e" and "%f" are specific text contents. The training content can also add corresponding training indicators based on the task characteristics of the rehabilitation training device to meet the training effect of the rehabilitation training device.

[0140] The multi-dimensional emotional interaction regulation strategy finally generated for the embodiment of the present disclosure can be completed with the help of external interactive devices. Specifically, the interpersonal interaction simulation in the multi-dimensional emotional interaction regulation strategy can be realized by external auditory and visual interactive devices, which are responsible for training motivational speech generation and virtual character image generation, respectively, to form auditory and visual interpersonal interaction simulation, and to enhance the positive emotions of rehabilitation of users by providing them with the positive effects of interpersonal interaction, thereby enhancing their motivation and involvement.

[0141] In summary, the embodiment of the present disclosure provides a method for generating an emotional interaction regulation strategy for rehabilitation training, which collects multi-source physiological and behavioral signals of users, discerns the psychological and physical state of users, and makes real-time decisions based on this with the help of a large language model to adjust the interaction strategy of the rehabilitation system. The system introduces the positive effects of interpersonal emotional interaction into the traditional human-computer interaction process of sports rehabilitation training dominated by rehabilitation robots, from personalized adjustment of task content to activate users' confidence and motivation in rehabilitation, to establishing a virtual character image to provide users with social emotional interaction and support with visual feedback and voice motivation. Therefore, the embodiment of the present disclosure can optimize the functions and effects of traditional rehabilitation systems, improve the effectiveness of rehabilitation training, help users improve their positive emotions and their compliance and participation in training, and conduct high-quality sports rehabilitation under the guidance of positive emotions, shorten the rehabilitation cycle, and improve the quality of life of users.

[0142] The second aspect of the present disclosure provides a device for generating an emotion interaction regulation strategy for rehabilitation training, comprising:

[0143] A user state analysis module is configured to obtain the user's psychological and physical state based on the collected physiological and behavioral signals of the user during the rehabilitation training process;

[0144] The emotion interaction regulation strategy generation module is configured to obtain the user's psychological and physical state, simulate the professional knowledge and thinking mode of rehabilitation physicians based on the large language model, understand the user's current state and make integrated reasoning decisions, and generate a multi-dimensional emotion interaction regulation strategy, including the training content of the rehabilitation training device, and the auditory and visual interaction content between the rehabilitation training device and the user that integrates interpersonal interaction factors.

[0145] It should be noted that the aforementioned explanation of the embodiment of the method for generating an emotion interaction regulation strategy for rehabilitation training is also applicable to the device for generating an emotion interaction regulation strategy for rehabilitation training of this embodiment, and will not be repeated here.

[0146] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0147] Although embodiments of the present disclosure have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and alterations may be made to the embodiments without departing from the principles and spirit of the present disclosure, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for generating an emotional interaction regulation strategy for rehabilitation training, characterized in that: include: Step S100, obtaining the user's psychological and physical state according to the collected physiological and behavioral signals of the user during the rehabilitation training process; Step S200: Based on the large language model, the professional knowledge and thinking mode of the rehabilitation physician are simulated to understand the user's current state and make integrated reasoning decisions, and generate a multi-dimensional emotional interaction regulation strategy, including the training content of the rehabilitation training device, and the auditory and visual interaction content between the rehabilitation training device and the user that integrates interpersonal interaction factors.

2. The method for generating an emotional interaction regulation strategy according to claim 1, characterized in that: In step S100, the process of obtaining the user's psychological state includes: preprocessing and preliminary feature extraction of various types of acquired signals respectively; then using a deep learning-based classification model to perform deep feature extraction on various types of extracted preliminary features to obtain the user's psychological state.

3. The method for generating an emotional interaction regulation strategy according to claim 2, characterized in that: The psychological state of the user is divided into an emotional state and a mental load level, the emotional state is divided into three levels: positive, neutral and negative, and the mental load level is divided into three levels: high, medium and low; The deep learning-based classification model is used to perform deep feature extraction on various extracted preliminary features to obtain the user's psychological state, specifically including: Each type of extracted preliminary features is taken as a modality, and a classification model based on deep learning is used to perform single-modal feature fusion and classification to obtain the preliminary psychological state classification results of the user; wherein, each modality uses a corresponding classification model, and during the training process of each classification model, a labeled data set is used for supervised learning, and the labels are evaluated based on the subjective emotion scale and the perceived stress scale. Through the scores of the subjects, the scores are divided into high, medium, and low training labels according to the scale standards to classify emotions and mental loads; Based on the rules, all the preliminary psychological state classification results of single modalities are subjected to multimodal decision fusion, wherein the rule is to dynamically adjust the weight of each modality in the decision fusion according to its individual classification performance, and the modality with better classification performance is given a higher weight. The weighted result is obtained by linearly weighted summing the classification labels of each modality, and the weighted result is normalized to obtain the user's final psychological state classification result as the discrimination result of the user's psychological state.

4. The method for generating an emotional interaction regulation strategy according to claim 1, characterized in that: In step S100, the user's physical condition is divided into peripheral fatigue degree and rehabilitation task performance; The peripheral fatigue degree is used to directly reflect the muscle state of the user, and is divided into three levels: high, medium, and low. The peripheral fatigue information is obtained according to the user's electromyographic signal, specifically including: preprocessing the electromyographic signal to obtain the peak electromyographic amplitude A of the current period, comparing the peak electromyographic amplitude A with the peak electromyographic amplitude A within the first 10 seconds of training. max When A / A max When it is between 80% and 100%, it is considered as low fatigue state. max Between 60% and 80%, it is considered a moderate fatigue state. max When it is 60% or below, it is considered as a high fatigue state; The performance of the rehabilitation task is related to the type and content of the rehabilitation training task. The evaluation indicators of the performance of the rehabilitation task include movement speed, movement smoothness, movement accuracy and evaluation indicators for the coordinated movement of bilateral limbs.

5. The method for generating an emotional interaction regulation strategy according to claim 1, characterized in that: Step S100 also includes: integrating the user's physical state and psychological state, and outputting them in a unified natural language format: "--Psychological state-- Emotional state: positive / neutral / negative; Mental workload: high / medium / low; --Physical Condition-- Peripheral fatigue: high / medium / low; Task performance: %a" Wherein, "%a" consists of specific text and a numerical value, indicating the name of the quantitative evaluation index of the rehabilitation training device and its corresponding numerical value.

6. The method for generating an emotional interaction regulation strategy according to claim 1, characterized in that: In step S200, the large language model is a pre-trained first large language model, and the pre-training process of the first large language model includes: Step S210, thinking chain training: performing the first fine-tuning training on the thinking chain of the first language model, by inputting typical cases of sports rehabilitation and theories of sports rehabilitation and disease psychology, training the first language model to gradually think and reason about the content of rehabilitation training and the ability to communicate content and attitude, wherein the rehabilitation training content includes training duration, training mode and task difficulty; Step S220, interactive text training: collect aging-friendly corpus and sports training motivational corpus to perform secondary fine-tuning training on the text content output by the first language model after the first fine-tuning, so that the interactive text output by the model is more suitable for the rehabilitation training scenario and meets user expectations.

7. The method for generating an emotional interaction regulation strategy according to claim 6, characterized in that: Step S210 specifically includes: Step S211: Collection of existing cases: Collect existing decision-making cases and decision-making ideas of licensed professional rehabilitation physicians when they provide rehabilitation training to users. An existing case i should at least record: (1) the user's condition s i , including injury time, lesion area and movement assessment; (2) initial artificial rehabilitation training program, which is equivalent to the initial rehabilitation training program adapted to the rehabilitation training device p i and the initial rehabilitation training program p i Normalized representation as a text sequence: "Training content: passive / assisted / active, resistance level, training speed, trajectory difficulty"; (3) The jth real-time adjustment strategy for training content and communication content during rehabilitation training a ij and reasons for adjustment ij , where strategy a will be adjusted ij The text normalization is represented as: "Training content adjustment: passive / assisted / active, resistance level, training speed, track difficulty; Communication content adjustment: communication attitude, communication text", where the communication attitude has two labels: "comfort" or "encouragement"; Adjustment reason r ij This includes changes in the user's mental state, physical state, and / or professional theoretical knowledge that the physician refers to when making adjustments; Step S212: Use the second language model to retrieve professional knowledge and expand the real-time adjustment reasons for the training content and communication content in the existing cases through reasoning analysis. ij : By designing prompt words, the second largest language model is prompted to retrieve professional rehabilitation theory and disease psychology theory. Based on the zero-sample thinking chain training method, instructions for the second largest language model to think step by step are added to the prompt words, so that the second largest language model can combine the retrieved knowledge to adjust the strategy a in the existing case i. ij Conduct step-by-step analytical reasoning to supplement and improve the reasons for adjustment ij , and the single adjustment strategy a ij Corresponding adjustment reason r ij The number of characters in the inference text is limited to not exceed the first maximum length Lr1; Step S213: Data standardization: All text s of case i i 、p i 、a ij 、r ij Generate text combination w according to the specified format i , and use all case text combinations to generate the first text sequence W = {w1,w2,…,w i ,…,w N }; Step S214: first fine-tuning training: The standardized first text sequence W is organized into a first fine-tuning dataset, and the first language model is trained based on the Hugging Face Transformers framework using the adapter fine-tuning method; wherein the first cross entropy loss function L is used. LOSS1 The adapter parameter θ1 is optimized as the objective function: In the formula, |r i | represents the total number of training adjustment strategy-adjustment reason groups for case i; P(p i |p i ,s i ; θ1) represents the first language model input user condition s after the first training i The output is then used to obtain the initial rehabilitation training program p i The probability of quantifying the ability to infer the initial training plan from the user's condition; P(a i,j |p i ,r i,≤j ,s i ,a i,<j θ1) is used to quantify the first language model after the first training. Based on the current user's condition, the initial rehabilitation training plan, the rehabilitation training adjustment strategy and adjustment reason before the jth adjustment, and the adjustment reason for the jth adjustment, the jth adjustment strategy a is inferred. i,j The probability of i,≤j represents the adjustment reason for the jth and previous rehabilitation training, a i,<j Represents the rehabilitation training adjustment strategy before the jth adjustment.

8. The method for generating an emotional interaction regulation strategy according to claim 6, characterized in that: Step S220 specifically includes: Step S221: Corpus collection: Collect aging-friendly corpus and rehabilitation exercise training motivational corpus from the Internet through web crawlers; Step S222: Data cleaning and preprocessing: The collected corpus is cleaned, including removing duplicate texts, correcting punctuation errors and grammatical confusion, and correcting typos and language irregularities in the corpus; the cleaned corpus is divided into different categories according to semantic functions, including motivational sentences, comforting sentences, task description sentences and daily communication sentences; the maximum number of characters in a single corpus segment is limited to not more than the second maximum length Lr2; all the obtained corpus segments are formatted into a second text sequence V = {v1, v2, ..., v l ,…,v M },v l Represents a single corpus fragment; Step S223: Second fine-tuning training The standardized second text sequence V is organized into the second fine-tuning dataset, and the first language model after the first fine-tuning is trained using the adapter fine-tuning method and the framework of Hugging Face Transformers; wherein the second cross entropy loss function L is used LOSS2 As the objective function to fine-tune the model parameters θ2: In the formula, P(v l ; θ2) represents the first language model prediction corpus segment v after secondary fine-tuning l probability.

9. The method for generating an emotional interaction regulation strategy according to claim 6, characterized in that: Step S200 also includes designing prompt words to enable the pre-trained first language model to generate standardized output, and the output content includes: "--Training content-- Training duration: %c minutes; Training mode: active / assisted / passive; Training impedance: %dN; --Virtual human interaction content-- Interaction text: %e; Interaction attitude: comfort / motivation. --Reason for adjustment-- ％f。” Among them, "%c" and "%d" are specific values, and "%e", "%f" and "%g" are specific text contents.

10. A device for generating an emotional interaction regulation strategy for rehabilitation training, characterized in that: include: A user state analysis module is configured to obtain the user's psychological and physical state based on the collected physiological and behavioral signals of the user during the rehabilitation training process; The emotion interaction regulation strategy generation module is configured to understand the user's current state and make integrated reasoning decisions based on the user's psychological and physical state by simulating the professional knowledge and thinking mode of rehabilitation physicians based on a large language model, and generate a multi-dimensional emotion interaction regulation strategy, including the training content of the rehabilitation training device, and the auditory and visual interaction content between the rehabilitation training device and the user that integrates interpersonal interaction factors.

Images