Virtual reality rehabilitation training adaptive regulation method and system based on flow double mapping model

Abstract

The application provides a virtual reality rehabilitation training adaptive regulation method and system based on a flow double mapping model, comprising: a pre-constructed flow double mapping model, which comprises a patient data and flow state input mapping model and a task parameter and flow state output mapping model; real-time acquisition of multi-dimensional data of a patient in virtual reality rehabilitation training; calculation of a current flow state by using the patient data and flow state input mapping model, and if the current flow state does not reach a target flow state, solving an optimal task parameter combination under the constraint condition of a medical rule by using the task parameter and flow state output mapping model, dynamically adjusting the virtual reality training task parameters, and until the flow state of the patient reaches a preset target. The application can precisely induce and maintain a higher flow state of the patient through multi-dimensional data fusion and flow double mapping model collaborative optimization, and significantly improve the participation and curative effect of rehabilitation training.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent medical treatment, and in particular to a virtual reality rehabilitation training adaptive regulation method and system based on a flow double mapping model. BACKGROUND

[0002] In recent years, virtual reality (VR) technology has shown great potential in the field of rehabilitation medicine. Its immersive environment can effectively improve the training participation and rehabilitation effect of patients. However, rehabilitation training usually needs to be persisted for a long time, and patients are prone to burnout due to repetitive training, resulting in decreased compliance. Psychological research shows that when an individual is in a "flow" state, they will show high concentration, control and internal motivation. This optimal experience state can significantly improve the training effect and patient persistence. Therefore, how to induce and maintain the flow state of patients in VR rehabilitation training has become a key problem to improve the rehabilitation effect.

[0003] Currently, some research has attempted to apply the flow theory to the VR rehabilitation field. The existing scheme one collects the electroencephalogram signals of the user, analyzes the relationship between the characteristics of a specific frequency band and the flow state, and adjusts the rhythm stimulation of the virtual scene accordingly. The existing scheme two uses the behavior performance data of the user, such as the action completion accuracy or the reaction time, to establish an association model between the task difficulty and the flow experience, and then realizes the dynamic adjustment of the task difficulty. These methods improve the training experience of the user to some extent.

[0004] However, the existing technology still has obvious deficiencies. First, the existing technology usually only relies on a single type of data input, such as using only physiological signals or only analyzing behavior performance, which cannot fully reflect the comprehensive state of the patient. Second, the regulation model is mostly a one-way mapping relationship, which fails to fully consider the synergistic effect between the patient characteristics and the task parameters. Third, the existing scheme lacks consideration of the special needs of the rehabilitation scene, such as the balance between medical safety constraints and personalized rehabilitation goals. These limitations make it difficult for the existing methods to achieve precise and stable flow state regulation in actual rehabilitation applications. SUMMARY

[0005] In view of this, the embodiments of the present application provide a virtual reality rehabilitation training adaptive regulation method and system based on a flow double mapping model to eliminate or improve one or more defects in the prior art, and solve the problems of single data dimension, simple regulation logic and insufficient rehabilitation adaptability in the prior art.

[0006] In one aspect, the present application provides a virtual reality rehabilitation training adaptive regulation method based on a flow double mapping model, which comprises the following steps:

[0007] real-time collection of multi-dimensional data of the patient in a virtual reality rehabilitation training task;

[0008] obtain a pre-constructed flow double-mapping model, the flow double-mapping model comprising a patient data and flow state input mapping model and a task parameter and flow state output mapping model; wherein the patient data and flow state input mapping model is used to represent a nonlinear mapping relationship between patient multi-dimensional data and a flow state, the patient multi-dimensional data at least comprising basic information of a patient, clinical evaluation data, and physiological signals, behavior performance data and subjective experience data in each virtual reality rehabilitation training task; the task parameter and flow state output mapping model is provided with a medical rule constraint condition, and is used to represent a mapping relationship between a task parameter combination of virtual reality rehabilitation training and a flow state;

[0009] In one regulation, the multi-dimensional data is input into the patient data and flow state input mapping model to calculate a current flow state;

[0010] If the current flow state does not reach a preset target flow state, an optimal parameter combination satisfying the medical rule constraint condition is solved by using the task parameter and flow state output mapping model; the task parameters in the virtual reality rehabilitation training task are regulated according to the optimal parameter combination; the regulation is repeated until the current flow state reaches the target flow state;

[0011] If the current flow state reaches the preset target flow state, the current flow state is maintained to continue the virtual reality rehabilitation training task.

[0012] In some embodiments of the present application, the method further comprises constructing the patient data and flow state input mapping model, the patient data and flow state input mapping model being obtained based on the following steps:

[0013] Collecting patient multi-dimensional data and pre-processing to construct a training set; labeling a real flow state label for the training set;

[0014] Constructing an initial model, the initial model adopting a multi-output machine learning model; the initial model taking patient data in the training set as input and outputting a predicted flow state;

[0015] Training the initial model by using the training set to minimize the error between the predicted flow state and the real flow state, optimizing the initial model, and finally obtaining the patient data and flow state input mapping model.

[0016] In some embodiments of the present application, the real flow state is quantitatively obtained based on the following steps:

[0017] The preset flow state scale is used to score the flow experience in each virtual reality rehabilitation training task in multiple dimensions to form a flow state vector as the real flow state; wherein, the flow state scale includes the following dimensions: challenge-skill balance, action-awareness fusion, clear goal, explicit feedback, concentration, sense of control, self-awareness weakening, time perception distortion, and internal motivation perception.

[0018] In some embodiments of the application, the method further comprises constructing a task parameter-flow state output mapping model based on the following steps:

[0019] A task parameter system is constructed, and a preset sampling method is used to generate a plurality of task parameter combinations for each task parameter in the task parameter system;

[0020] A state space is constructed using the task parameter combinations, real-time state data of the patient, and the flow state, an action space is constructed for the adjustment amount of the task parameters, and a reward function is constructed based on the flow state improvement amount and the medical rule constraint condition;

[0021] An Actor-Critic network is used to construct a reinforcement learning model, a state vector of the state space is input into the Actor network to generate an action probability distribution, and the state vector is input into the Critic network to generate a state value estimate; a mean square error loss is constructed based on the state value estimate, and the parameters of the Actor network and the Critic network are updated to minimize the mean square error loss, and finally the task parameter-flow state output mapping model is obtained.

[0022] In some embodiments of the application, the task parameter system includes a plurality of adjustable task parameters, including at least avatar parameters, narrative parameters, task mechanism parameters, aesthetic parameters, music parameters, and multi-modal stimulation parameters.

[0023] In some embodiments of the application, the medical rule constraint condition includes but is not limited to a motor function constraint condition, a training dose constraint condition, and a safety threshold constraint condition; wherein, the motor function constraint condition is used to limit the adjustment range of the task parameters to not exceed the bearable threshold of the current physiological function of the patient; the training dose constraint condition is used to control the intensity of the virtual reality rehabilitation training to comply with the clinical treatment specification; and the safety threshold constraint condition is used to ensure that the intensity of the sensory stimulation does not exceed the individualized tolerance limit of the patient.

[0024] In some embodiments of the application, the task parameter-flow state output mapping model is used to solve the optimal parameter combination that satisfies the medical rule constraint condition, including:

[0025] calculating a difference value between the current flow state and the target flow state;

[0026] calculating a contribution degree of each task parameter to the flow state improvement based on a back propagation gradient of the task parameter and the flow state output mapping model, and generating an adjustment priority sequence;

[0027] screening out a preset number of task parameters with the largest contribution degree as optimization variables according to the adjustment priority sequence;

[0028] optimizing the optimization variables by using an iterative optimization algorithm to minimize the difference value in a feasible solution space meeting the medical rule constraint condition, and obtaining the optimal parameter combination;

[0029] In one iteration, the iterative optimization algorithm performs the following operations: generating a candidate parameter combination according to a current gradient direction; inputting the candidate parameter combination meeting the medical rule constraint condition into the task parameter and flow state output mapping model to obtain a predicted flow state; calculating a new difference value between the predicted flow state and the target flow state; if the new difference value is smaller than the original difference value, updating the optimal parameter combination; and iterating until a preset iteration number is reached or the new difference value converges.

[0030] In some embodiments of the present application, the task parameters in the virtual reality rehabilitation training task are regulated according to the optimal parameter combination, including collaborative adjustment of at least two types of task parameters, wherein the adjustment mode of each task parameter includes:

[0031] adjusting the avatar parameters, including adjusting one or more of the limb proportion, the movement range and the action guide marker of the virtual avatar;

[0032] adjusting the narrative parameters, including adjusting one or more of the task progress, the task difficulty and the task environment story element;

[0033] adjusting the task mechanism parameters, including adjusting one or more of the virtual obstacle attribute, the task time limit and the interaction mechanism;

[0034] adjusting the aesthetic parameters, including adjusting one or more of the color contrast of the virtual scene and the visual saliency of the interface element;

[0035] adjusting the music parameters, including adjusting one or more of the background music type, the rhythm and the prompt audio frequency;

[0036] adjusting the multi-modal stimulation parameters, including adjusting one or more of the multi-sensory feedback intensity and the environmental interference element.

[0037] In another aspect, the present application also provides a virtual reality rehabilitation training adaptive regulation system based on a flow double mapping model, which, when executed, implements the steps of the method according to any one of the above aspects, and comprises:

[0038] a model construction module for constructing and maintaining a flow double mapping model;

[0039] a closed-loop regulation module for regulating virtual reality rehabilitation training task parameters in real time based on the flow double mapping model to make the patient's flow state reach a preset target flow state;

[0040] a virtual reality training module comprising a task scene construction unit and a human-computer interaction interface for executing and presenting virtual reality rehabilitation training tasks.

[0041] In another aspect, the present application also provides a computer readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the steps of the method according to any one of the above aspects.

[0042] The present application provides a virtual reality rehabilitation training adaptive regulation method and system based on a flow double mapping model, comprising: a pre-constructed flow double mapping model comprising a patient data and flow state input mapping model and a task parameter and flow state output mapping model; real-time acquisition of multi-dimensional data of the patient in virtual reality rehabilitation training; calculation of the current flow state by using the patient data and flow state input mapping model, and solving of the optimal task parameter combination under the constraint condition of medical rules by using the task parameter and flow state output mapping model if the current flow state does not reach the target flow state, dynamic adjustment of the virtual reality training task parameters until the patient's flow state reaches the preset target. The present application can precisely induce and maintain a higher flow state of the patient through multi-dimensional data fusion and flow double mapping model collaborative optimization, significantly improve the participation and curative effect of rehabilitation training. At the same time, it supports flexible configuration of data acquisition methods (such as wearable devices or VR interaction data), reduces the dependence on a single device, is suitable for various scenes such as motor function recovery and cognitive training, and has individualized adaptability and medical safety.

[0043] Additional advantages, objects, and features of the application will be set forth in part by the description that follows, and will in part be apparent to those of ordinary skill in the art upon examination of the following or can be learned from a practice of the application. The advantages and objects of the application can be realized and obtained by means of the instrumentalities and combinations pointed out in the appended description.

[0044] Those skilled in the art will appreciate that the objects and advantages of the application can be realized and obtained by means of the instrumentalities and combinations pointed out in the appended description. Attached Figure Description

[0045] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, are not intended to limit the scope of the invention. In the drawings:

[0046] Figure 1 This is a schematic diagram of the steps of an adaptive control method for virtual reality rehabilitation training based on a flow dual-mapping model in one embodiment of the present invention.

[0047] Figure 2 This is a schematic diagram illustrating the construction of a flow dual-mapping model in one embodiment of the present invention.

[0048] Figure 3 This is a flowchart of a virtual reality rehabilitation training adaptive control method in one embodiment of the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0050] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0051] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0052] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0053] In the following description, embodiments of the invention will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0054] It should be emphasized here that the step markers mentioned below are not a limitation on the order of the steps, but should be understood as meaning that the steps can be executed in the order mentioned in the embodiments, or in a different order than in the embodiments, or several steps can be executed simultaneously.

[0055] In order to solve the problems of single data dimension, simple regulation logic and insufficient rehabilitation adaptability in the prior art, the present application provides a virtual reality rehabilitation training self-adaptive regulation method based on a flow double mapping model, as shown in Figure 1 The method comprises the following steps S101-S105:

[0056] Step S101: Real-time collection of multi-dimensional data of a patient in a virtual reality rehabilitation training task.

[0057] Step S102: Obtaining a pre-constructed flow double mapping model, wherein the flow double mapping model comprises a patient data and flow state input mapping model and a task parameter and flow state output mapping model. The patient data and flow state input mapping model is used to represent the nonlinear mapping relationship between the patient multi-dimensional data and the flow state, and the patient multi-dimensional data at least comprises the basic information of the patient, the clinical evaluation data, and the physiological signal, the behavior performance data and the subjective experience data in each virtual reality rehabilitation training task. The task parameter and flow state output mapping model is provided with a medical rule constraint condition, and is used to represent the mapping relationship between the task parameter combination of the virtual reality rehabilitation training and the flow state.

[0058] Step S103: In one regulation, the multi-dimensional data is input into the patient data and flow state input mapping model, and the current flow state is calculated.

[0059] Step S104: If the current flow state does not reach the preset target flow state, the optimal parameter combination satisfying the medical rule constraint condition is solved by using the task parameter and flow state output mapping model; the task parameters in the virtual reality rehabilitation training task are regulated according to the optimal parameter combination; the regulation is repeated until the current flow state reaches the target flow state.

[0060] Step S105: If the current flow state reaches the preset target flow state, the current flow state is maintained to continue the virtual reality rehabilitation training task.

[0061] In the virtual reality rehabilitation training self-adaptive regulation method based on the flow double mapping model, the pre-constructed flow double mapping model is needed, therefore, the flow double mapping model is first described to facilitate understanding.

[0062] As shown in Figure 2 It is a construction schematic diagram of the flow double mapping model.

[0063] The flow double mapping model comprises a patient data and flow state input mapping model and a task parameter and flow state output mapping model. The patient data and flow state input mapping model is used to represent a nonlinear mapping relationship between patient multidimensional data and a flow state, and the task parameter and flow state output mapping model is used to represent a mapping relationship between a task parameter combination of virtual reality rehabilitation training and a flow state.

[0064] The flow state is a comprehensive experience comprising challenge-skill balance, behavior-conscious fusion, clear goal, clear feedback, concentration, sense of control, self-awareness weakening, time perception distortion and internal motivation perception, and can be obtained based on a score result of a flow state scale, such as a Flow State Scale-2 scale.

[0065] It should be noted that the regulation of a single task parameter cannot comprehensively cover the multidimensional optimization requirements of the flow state. For example, the challenge-skill balance dimension mainly depends on the adjustment of a task mechanism parameter (such as obstacle difficulty); the concentration dimension is more susceptible to the influence of a music parameter (such as music rhythm); and the sense of control dimension needs to be enhanced through an avatar parameter (such as a motion guide marker). Therefore, in the present application, a task parameter and flow state output mapping model is constructed, and through the mapping of multidimensional task parameters and multidimensional flow states, the interrelated flow dimensions are simultaneously optimized through joint adjustment of multiple task parameters, such as increasing the obstacle height while increasing the auxiliary marker, which enhances the challenge and guarantees the sense of control; at the same time, the problem of negative interference between dimensions caused by traditional single-parameter regulation is avoided, for example, the problem that increasing the task difficulty may cause a decrease in the sense of control.

[0066] Multidimensional data of the patient is collected, wherein the multidimensional data comprises basic information of the patient, clinical evaluation data, and physiological signals, behavior performance data and subjective experience data in each virtual reality rehabilitation training task. Specifically,

[0067] The basic information comprises basic information of the patient, such as age, gender, medical history, etc.

[0068] The clinical evaluation data is the clinical functional evaluation data of the patient, comprising motor function evaluation data, daily activity function evaluation data, cognitive function evaluation data, etc. The motor function evaluation can be evaluated by using a Fugl-Meyer scale, a Berg balance scale, etc.; and the cognitive function evaluation can be evaluated by using a MoCA (Montreal Cognitive Assessment), an MMSE (Mini-Mental State Examination), etc.

[0069] The physiological signals comprise heart rate, skin conductivity, breathing rate, facial expression, blood oxygen saturation, electroencephalogram, eye movement data, etc. of the patient collected in real time in the virtual reality rehabilitation training task. The physiological signals are potential important physiological characteristics that can map the flow state.

[0070] The behavior performance data includes the action completion degree (such as the degree of joint activity, action standardization, etc.), reaction time, task completion time length, etc. of the patient collected in real time in the virtual reality rehabilitation training task. The behavior performance data can reflect the skill performance of the patient in the training and the adaptation to the task.

[0071] The subjective experience data includes the scores of the patient's subjective feelings in terms of fatigue, concentration, control, etc. of the current training task. Exemplarily, a flow experience questionnaire can be presented to the patient through an interactive interface in the virtual reality scene for scoring, wherein the flow experience questionnaire can use the Flow State Scale-2 scale to obtain multi-dimensional patient subjective flow experience data.

[0072] The collected multi-dimensional data of the patient is cleaned and normalized to remove noise and outliers, ensuring the accuracy and usability of the data.

[0073] Based on the multi-dimensional data of the patient processed as described above, a patient data and flow state input mapping model and a task parameter and flow state output mapping model are constructed.

[0074] In some embodiments, the construction (training) method of the patient data and flow state input mapping model includes:

[0075] A training set is constructed, including collecting and preprocessing the multi-dimensional data of the patient. Exemplarily, a timestamp synchronization technology is used to unify data of different sampling frequencies to the same time reference (such as 100 Hz for physiological signals and 30 Hz for behavior data), and alignment is achieved through interpolation or down-sampling, and synchronization control error is controlled.

[0076] Each sample in the training set is labeled with a true flow state label. A preset flow state scale is used to score the flow experience in each virtual reality rehabilitation training task in multiple dimensions to form a flow state vector as the true flow state. Exemplarily, the preset flow state scale uses the Flow State Scale-2 scale, which includes the following 9 dimensions: challenge-skill balance, behavior-conscious fusion, clear goal, clear feedback, concentration, control, self-awareness weakening, time perception distortion, and internal motivation perception.

[0077] An initial model is constructed, which uses a multi-output machine learning model such as a tensor regression network or a graph neural network. The initial model takes the multi-dimensional data of the patient as input and outputs the corresponding predicted flow state.

[0078] The initial model is trained by using the training set, a weighted mean square loss between the predicted flow state and the real flow state is constructed, and the initial model is optimized to minimize the loss, and finally a patient data and flow state input mapping model is obtained.

[0079] It should be noted that in the patient data and flow state input mapping model, the physiological signals (heart rate, electroencephalogram, etc.), behavioral performance data (motion accuracy, reaction time, etc.), and subjective experience (fatigue, interest, etc.) of the patient are multi-dimensional data, and the association with the flow state has specificity, such as heart rate changes mainly affecting the "sense of control" dimension, and motion accuracy is strongly correlated with the "challenge-skill balance" dimension. Therefore, if only a single-dimensional flow indicator is used, it is impossible to establish an accurate mapping with the patient's multi-dimensional data. At the same time, some patient data types may only be strongly correlated with certain dimensions of the flow state and have no correlation with the overall flow. For example, the electroencephalogram signal may only be strongly correlated with the "focus" dimension and have no correlation with the overall flow score, but this does not mean that the electroencephalogram signal is not related to the flow state, but rather that the electroencephalogram signal can reflect the dimensional characteristics of the flow state. Therefore, if only the overall flow score is used to map the electroencephalogram signal, the mapping relationship between the electroencephalogram signal and the "focus" dimension is lost, and it is impossible to establish a precise mapping with the patient's multi-dimensional data.

[0080] Similarly, based on the collected multi-dimensional data of the patient, especially the physiological signals, behavioral performance data, and subjective experience data in each virtual reality rehabilitation training task, a task parameter and flow state output mapping model is constructed by using a back propagation neural network or a reinforcement learning model.

[0081] In some embodiments, the construction (training) method of the task parameter and flow state output mapping model includes:

[0082] A task parameter system is constructed, including a variety of adjustable task parameters: avatar parameters, narrative parameters, task mechanism parameters, aesthetic parameters, music parameters, and multi-modal stimulation parameters. Among them, the avatar parameters include limb proportions, motion ranges, etc., the narrative parameters include task difficulty, story progress, etc., the task mechanism parameters include obstacle attributes, task time limits, etc., the aesthetic parameters include color contrast, etc., the music parameters include music rhythm, volume, etc., and the multi-modal stimulation parameters include tactile feedback intensity, etc.

[0083] Intelligent sampling methods such as Latin hypercube sampling method are used for each task parameter to generate a variety of task parameter combinations.

[0084] An reinforcement learning model is constructed, which adopts an Actor-Critic network; a state space is constructed with current task parameters, real-time state data of the patient (including physiological signals and behavior performance data, etc.) and current flow state; an action space is constructed with adjustment amount of the task parameters; a reward function is constructed based on flow state improvement amount and medical rule constraint conditions. The flow state improvement amount is obtained by weighted summation of 9 dimensions.

[0085] The state vector of the state space is input into the Actor network to generate an action probability distribution; the state vector is input into the Critic network to generate a state value estimate. Based on the state value estimate, a mean square error loss is constructed, and the parameters of the Actor network and the Critic network are updated to minimize the mean square error loss, so that the model learns the mapping relationship between the task parameter combination and the flow state, outputs the optimal task parameter adjustment scheme, and finally obtains a task parameter and flow state output mapping model.

[0086] Among them, different patients, due to their age, disease, gender and other basic information, have different tolerances and sensitivities to task parameters, such as elderly patients having lower tolerance to complex tasks than young patients, and elderly patients having lower sensitivity to music rhythm than young patients. Through the mapping of multi-dimensional task parameters and flow state, the corresponding task parameter combination can be matched for the short board dimension of the patient's flow, such as adjusting the prompt audio frequency and interface feedback intensity when the "clear feedback" dimension is insufficient; or, such as the target threshold of "challenge-skill balance" dimension of patients in the early stage of stroke being lower than that of patients in the recovery period, etc., to achieve personalized adaptation.

[0087] In some embodiments, the medical rule constraint conditions include:

[0088] Motion function constraint conditions are used to limit the adjustment range of the task parameters to be within the tolerable threshold of the patient's current physiological function, such as the adjustment range of the obstacle being less than or equal to the joint activity threshold of the patient.

[0089] Training dose constraint conditions are used to control the virtual reality rehabilitation training intensity to comply with clinical treatment specifications, such as the task time limit complying with the clinical treatment guidelines, the single training duration being no more than 30 minutes, and the total training duration per day being no more than 1 hour.

[0090] Safety threshold constraint conditions are used to ensure that the intensity of sensory stimulation does not exceed the individual tolerance limit of the patient, such as the intensity of tactile feedback not exceeding 50% of the patient's pain threshold.

[0091] In some embodiments, the medical rule constraint conditions are embedded into the task parameter and flow state output mapping model, including:

[0092] For motion function constraints, hard constraints can be applied by adding parameter range limiting functions to the model output layer.

[0093] For training dose constraints, soft constraints can be implemented by adding a penalty term to the reinforcement learning reward function. For example, for training duration constraints, a time penalty term can be added to the reinforcement learning reward function. When the cumulative training time exceeds the preset duration, the reward value drops sharply.

[0094] For safety threshold constraints, a safety activation function can be added to the last layer of the Actor network to constrain the network structure.

[0095] When solving for the optimal parameter combination based on the task parameter and flow state output mapping model, the improvement of multiple flow dimensions can be balanced within the constraints of the above medical rules. For example, while limiting the height of the obstacle, the "behavioral consciousness integration" dimension can be enhanced through aesthetic parameters to avoid exceeding the safety threshold when adjusting a single parameter.

[0096] like Figure 3 The diagram shows a flowchart of the adaptive control method for virtual reality rehabilitation training based on the flow dual mapping model constructed above.

[0097] In step S101, the virtual reality rehabilitation equipment is activated, and the training task and parameters are determined based on the patient's information to conduct rehabilitation training. Multidimensional data of the patient during the training process are collected in real time, such as physiological signals, behavioral data, and subjective experience data, obtained through wearable devices, VR controllers, eye trackers, etc.

[0098] In step S102, the pre-constructed flow dual mapping model is obtained.

[0099] In step S103, the patient's multidimensional data collected in step S101 is input into the patient data in the heart flow dual mapping model and the heart flow state is input into the mapping model to obtain the patient's current heart flow state.

[0100] In step S104, the patient's current cardiac flow state is compared with the preset target cardiac flow state to determine whether the task parameters need to be adjusted.

[0101] If the current flow state does not reach the preset target flow state, the optimal parameter combination that satisfies the medical rule constraints is solved using the task parameter and flow state output mapping model; the task parameters in the virtual reality rehabilitation training task are adjusted according to the optimal parameter combination.

[0102] In some embodiments, finding the optimal combination of parameters that satisfies the constraints of medical rules includes the following steps:

[0103] The current flow state is subtracted from the target flow state to calculate the difference value of each dimension. According to the preset allowable error threshold, the dimension set that needs to be optimized, i.e., the target dimension set, is screened out.

[0104] By back propagation of the task parameter and flow state output mapping model, the gradient of each task parameter to the target dimension is calculated, and the adjustment priority sequence is generated in descending order of absolute value of the gradient.

[0105] From the adjustment priority sequence, Top-K parameters are selected as optimization variables, and the remaining parameters are fixed as the current values. For example, K=3.

[0106] In the feasible solution space defined by the medical rule constraint condition, the optimization variables are optimized by an iterative optimization algorithm to minimize the difference between the current flow state and the target flow state, and the optimal parameter combination is obtained.

[0107] In some embodiments, the iterative optimization algorithm optimization step includes:

[0108] The maximum number of iterations and the convergence threshold are set, the current parameter combination is taken as the optimal solution, and the current difference value is taken as the initial difference value.

[0109] In each iteration, based on the current parameters and gradient information, a candidate parameter combination is generated along the negative gradient direction. The candidate parameter combination is checked for medical rule constraints item by item. The candidate parameter combination that passes the check is input into the task parameter and flow state output mapping model to obtain the predicted flow state. The new difference value between the predicted flow state and the target flow state is calculated. If the new difference value is less than the original difference value, the candidate parameter combination is updated as the current optimal solution. The iteration is performed until the preset maximum number of iterations is reached or the new difference value is less than the preset convergence threshold, and the optimal solution, i.e., the optimal parameter combination, is output. For example, an iteration interval time is set, such as 500ms.

[0110] Based on the optimal parameter combination, the task parameters in the virtual reality rehabilitation training task are regulated, and the precise maintenance of the flow state is finally realized, and the participation and efficacy of rehabilitation training are improved.

[0111] In some embodiments, the adjustment of each task parameter includes:

[0112] The avatar parameters are adjusted, including adjusting one or more of the limb proportions, movement ranges, and action guide markers of the virtual avatar. For example, adjusting the scaling factor to adapt to the patient's movement ability, visualizing the joint range of motion, dynamically highlighting the key muscle group force points, etc.

[0113] Adjusting narrative parameters, including adjusting one or more of task progress, task difficulty, and task environment story elements. For example, adjusting task level unlock conditions, subtask quantity and task completion time limit, setting scene details related to patient life, etc.

[0114] Adjusting task mechanism parameters, including adjusting one or more of virtual obstacle attributes, task time limit, and interaction mechanism. For example, adjusting the vertical height of virtual steps and trenches, the countdown threshold for completing the target, the action response sensitivity and the content and frequency of error correction prompts.

[0115] Adjusting aesthetic parameters, including adjusting one or more of color contrast of virtual scenes and visual saliency of interface elements. For example, highlighting the key path with high-contrast colors, selecting weather environments such as rainy, sunny, snowy, etc., adjusting the transparency of UI interface (visual weight of progress bar, prompt box), etc.

[0116] Adjusting music parameters, including adjusting one or more of background music type, rhythm, and prompt audio frequency. For example, adding voice guidance, selecting instrument timbre, adjusting the interval between sound effects of key actions, the synchronization coefficient of background music rhythm and action frequency, etc.

[0117] Adjusting multi-modal stimulation parameters, including adjusting one or more of multi-sensory feedback intensity and environmental interference elements. For example, adjusting the vibration amplitude of the vibrating vest, the degree of randomization of virtual object movement, releasing specific smells according to specific scenes, and adjusting the concentration of the smell.

[0118] In step S105, if the current flow state reaches the preset target flow state, the current flow state is maintained for rehabilitation training, and no adjustment is made.

[0119] Corresponding to the above method, the present application also provides a virtual reality rehabilitation training adaptive control system based on a flow double-mapping model. The system, when executed, can implement the steps of the above method. The system comprises:

[0120] A model construction module for constructing and maintaining a flow double-mapping model.

[0121] A closed-loop control module for real-time adjustment of virtual reality rehabilitation training task parameters based on the flow double-mapping model to make the patient's flow state reach the preset target flow state.

[0122] A virtual reality training module, including a task scene construction unit and a human-computer interaction interface, for executing and presenting virtual reality rehabilitation training tasks.

[0123] The present application will be further described below in conjunction with a specific embodiment.

[0124] In this embodiment, the age, gender and other basic information of 100 stroke patients and the clinical evaluation data such as Fugl-Meyer motor scale are collected, the heart rate, respiratory rate and other physiological signals of the patients in different VR walking tasks are synchronously obtained, and the step length, joint range of motion, obstacle avoidance success rate and other behavior performance data are obtained, and the subjective scores of 9 dimensions including challenge-skill balance, concentration, sense of control and the like are collected through the Flow State Scale-2 questionnaire embedded in VR. Based on the collected information, a patient data and flow state input mapping model is constructed, it is found that the step length difference and the challenge-skill balance dimension, and the heart rate and the sense of control dimension have significant correlation; at the same time, a task parameter and flow state output mapping model is constructed, and the influence weight of the obstacle height on the challenge-skill balance dimension and the influence weight of the music rhythm on the concentration dimension are determined.

[0125] In the adaptive regulation method, the system gives the initial task parameters as the obstacle height 10 cm and the music rhythm 80 bpm. The current collected step length difference 12 cm and heart rate 95 times / minute are input into the patient data and flow state input mapping model, and the current flow state is predicted as: the challenge-skill balance dimension 4 points, the sense of control dimension 5 points (total score 40 points), which is significantly lower than the preset target flow state (each dimension ≥ 6 points, total score 60 points). The system calculates the optimal parameter combination based on the task parameter and flow state output mapping model to adjust the task parameters: the obstacle height is reduced to 8 cm to optimize the challenge-skill balance dimension, and the music rhythm is adjusted to 60 bpm to improve the concentration dimension. After adjusting the task parameters, the patient's step length difference is reduced to 8 cm, the heart rate is reduced to 78 times / minute, and the flow score is increased to 66 points (challenge-skill balance 7 points, sense of control 7 points, and the remaining dimensions ≥ 6 points), and the data collection-state evaluation-parameter adjustment closed loop is completed every 300 ms, so that the patient can continuously be in a high flow state to improve the walking training compliance.

[0126] The application will be further described below in combination with a specific embodiment.

[0127] In this embodiment, the basic information and Berg balance scale scores and other clinical data of 80 Parkinson's patients are collected, the heart rate, skin conductivity and other physiological signals of the patients in VR tasks are synchronously obtained, and the subjective scores of 9 dimensions are collected through the 9-dimension flow questionnaire. Based on the collected data, a patient data and flow state input mapping model is constructed, it is found that the patient's center of gravity offset amplitude and the sense of control dimension, and the heart rate variability (HRV) and the intrinsic motivation dimension have significant negative correlation, and a task parameter and flow state output mapping model is constructed, and the influence weight of the virtual obstacle moving speed on the challenge-skill balance dimension and the influence weight of the scene ground texture complexity on the concentration dimension are determined.

[0128] In the adaptive regulation method, the system gives the initial task parameters as the virtual obstacle moving speed 1.0 m / s and the ground texture complexity level 3. The patient data with the current collected patient center of gravity offset amplitude reaching 4 cm and the reduced HRV are input into the flow state input mapping model, and the current flow state is predicted as: "sense of control" 5 points, "concentration" 6 points (total score 70 points), which is lower than the target threshold (each dimension ≥ 7 points, total score 70 points). The system calculates the optimal parameter combination based on the task parameter and flow state output mapping model to adjust the task parameters: the obstacle moving speed is reduced to 0.8 m / s to improve the "sense of control" dimension, and the ground texture is simplified to level 2 to reduce the cognitive load, thereby improving the "concentration" dimension. After adjustment, the patient center of gravity offset amplitude is reduced to 2.5 cm, the HRV is restored to normal, and the flow score is increased to 80 points ("sense of control" 7 points, "concentration" 7 points). The closed-loop regulation is completed once every 400 ms, and the effect of balance function training of Parkinson's patients is improved through precise adaptation of the task parameters.

[0129] Corresponding to the above method, the application also provides an electronic device, which comprises a computer device, the computer device comprising a processor and a memory, the memory storing computer instructions, and the processor being configured to execute the computer instructions stored in the memory, so that the electronic device implements the steps of the method as described above.

[0130] The embodiment of the application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by the processor to implement the steps of the above method. The computer readable storage medium can be a tangible storage medium, such as random access memory (RAM), internal memory, read only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable memory disk, CD-ROM, or any other form of storage medium known in the technical field.

[0131] Those of ordinary skill in the art will appreciate that the various illustrative components, systems and methods described in connection with the embodiments disclosed herein can be implemented as hardware, software, or both. The particular implementation is dependent on the specific application and design constraints imposed on the overall system. Skilled persons can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application. When implemented in hardware, for example, the hardware can comprise an electronic circuit, an Application Specific Integrated Circuit (ASIC), a suitable firmware, a plug-in, a functional card, etc. When implemented in software, the elements of the application are the program or code segments to perform a specific task. The program or code segments can be stored in a machine-readable medium, or transmitted by a carrier wave as data signals over a transmission medium or communication link.

[0132] It is to be understood that the application is not limited to the particular configurations and processes described herein and shown in the drawings. For simplicity, detailed descriptions of known methods and apparatuses are omitted so as not to obscure the disclosure. In the above-described embodiments, several specific steps are described and illustrated as examples. However, the method processes of the present application are not limited to the specific steps described and illustrated, and the order of the steps can be changed, or other steps can be added, or replaced, or eliminated, depending on the application.

[0133] In the present application, features described and / or illustrated in relation to one embodiment can be used in the same or a similar way in one or more other embodiments, and / or combined with or instead of features of other embodiments.

[0134] The above description is only preferred embodiments of the present application, and is not intended to limit the present application. The embodiments of the present application can be variously changed and modified by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the present application.

Claims

1. A virtual reality rehabilitation training adaptive regulation method based on a flow double-mapping model, characterized in that, The method comprises: real-time acquisition of multi-dimensional data of a patient in a virtual reality rehabilitation training task; obtaining a pre-constructed flow double mapping model, the flow double mapping model comprising a patient data and flow state input mapping model and a task parameter and flow state output mapping model; wherein the patient data and flow state input mapping model is used to represent the nonlinear mapping relationship between the patient multi-dimensional data and the flow state, and the patient multi-dimensional data at least comprises the basic information, clinical evaluation data of the patient, and physiological signals, behavior performance data and subjective experience data in each virtual reality rehabilitation training task; the task parameter and flow state output mapping model is provided with a medical rule constraint condition, and is used to represent the mapping relationship between the task parameter combination of the virtual reality rehabilitation training and the flow state; in one regulation, the multi-dimensional data is input into the patient data and flow state input mapping model, and the current flow state is calculated; if the current flow state does not reach the preset target flow state, the optimal parameter combination satisfying the medical rule constraint condition is solved by using the task parameter and flow state output mapping model; the task parameters in the virtual reality rehabilitation training task are regulated according to the optimal parameter combination; the regulation is repeated until the current flow state reaches the target flow state; if the current flow state reaches the preset target flow state, the current flow state is maintained to continue the virtual reality rehabilitation training task; wherein the task parameter and flow state output mapping model is obtained based on the following steps: constructing a task parameter system, using a preset sampling method for each task parameter in the task parameter system to generate a plurality of task parameter combinations, the task parameters at least comprising avatar parameters, narrative parameters, task mechanism parameters, aesthetic parameters, music parameters and multi-modal stimulation parameters; constructing a state space with the task parameter combination, patient real-time state data and flow state, constructing an action space with the adjustment amount of the task parameter, constructing a reward function based on the flow state improvement amount and the medical rule constraint condition; using an Actor-Critic network to construct a reinforcement learning model, inputting the state vector of the state space into the Actor network to generate an action probability distribution, inputting the state vector into the Critic network to generate a state value estimate; constructing a mean square error loss based on the state value estimate, updating the parameters of the Actor network and the Critic network to minimize the mean square error loss, and finally obtaining the task parameter and flow state output mapping model.

2. The method of claim 1, wherein, The method further comprises constructing the patient data and flow state input mapping model, and the patient data and flow state input mapping model is obtained based on the following steps: acquiring and preprocessing the patient multi-dimensional data to construct a training set; labeling the real flow state label for the training set; constructing an initial model, the initial model using a multi-output machine learning model; the initial model takes the patient data in the training set as input and outputs the predicted flow state; The initial model is trained by using the training set, and the initial model is optimized to minimize the error between the predicted flow state and the real flow state, and finally the patient data and flow state input mapping model is obtained.

3. The method of claim 2, wherein, The real flow state is quantified based on the following steps: A preset flow state scale is used to score the flow experience in each virtual reality rehabilitation training task in multiple dimensions to form a flow state vector as the real flow state; wherein the flow state scale includes the following dimensions: challenge-skill balance, behavior-consciousness integration, clear goal, clear feedback, concentration, sense of control, self-awareness weakening, time perception distortion, and internal motivation perception.

4. The method of claim 1, wherein, The medical rule constraint conditions include but are not limited to motion function constraint conditions, training dose constraint conditions, and safety threshold constraint conditions; wherein the motion function constraint condition is used to limit the regulation range of the task parameter to not exceed the bearable threshold of the current physiological function of the patient; the training dose constraint condition is used to control the virtual reality rehabilitation training intensity to comply with the clinical treatment specification; and the safety threshold constraint condition is used to ensure that the sensory stimulation intensity does not exceed the individualized tolerance limit of the patient.

5. The method of claim 1, wherein, The optimal parameter combination satisfying the medical rule constraint condition is solved by using the task parameter and flow state output mapping model, including: calculating the difference between the current flow state and the target flow state; based on the back propagation gradient of the task parameter and flow state output mapping model, calculating the contribution degree of each task parameter to the flow state improvement, and generating an adjustment priority sequence; According to the adjustment priority sequence, the first preset number of task parameters with the largest contribution degree are selected as optimization variables; In the feasible solution space satisfying the medical rule constraint condition, the optimization variables are optimized by using an iterative optimization algorithm to minimize the difference, and the optimal parameter combination is obtained; Wherein, the iterative optimization algorithm performs the following operations in one iteration: generating a candidate parameter combination according to the current gradient direction; inputting the candidate parameter combination satisfying the medical rule constraint condition into the task parameter and flow state output mapping model to obtain the predicted flow state; calculating the new difference between the predicted flow state and the target flow state; if the new difference is less than the original difference, updating the optimal parameter combination; iterating until a preset iteration number or the new difference converges.

6. The method of claim 1, wherein, According to the optimal parameter combination, the task parameters in the virtual reality rehabilitation training task are regulated, including the coordinated adjustment of at least two types of task parameters, wherein the adjustment mode of each task parameter includes: adjusting the avatar parameter, including adjusting one or more of the virtual avatar's body proportion, movement range and action guidance marker; adjusting the narrative parameter, including adjusting one or more of the task progress, task difficulty and task environment story element; adjusting the task mechanism parameter, including adjusting one or more of the virtual obstacle attribute, task time limit and interaction mechanism; adjusting the aesthetic parameters includes adjusting one or more of color contrast of the virtual scene, visual saliency of interface elements; adjusting the music parameters includes adjusting one or more of background music type, rhythm, and cue audio frequency; adjusting the multi-modal stimulation parameters includes adjusting one or more of multi-sensory feedback intensity and environmental distractor elements.

7. A virtual reality rehabilitation training adaptive regulation system based on a flow double-mapping model, characterized in that, The system, when executed, implements the steps of the method of any one of claims 1-6, the system comprising: a model building module for building and maintaining a flow double-mapping model; a closed-loop regulation module for regulating virtual reality rehabilitation training task parameters in real time based on the flow double-mapping model to make the patient's flow state reach a preset target flow state; a virtual reality training module including a task scene building unit and a human-computer interaction interface for executing and presenting virtual reality rehabilitation training tasks.

8. A computer readable storage medium having stored thereon computer programs / instructions, characterized in that, The computer program / instructions, when executed by a processor, implement the steps of the method of any one of claims 1-6.

Images