Children hand function rehabilitation system based on game interaction

Abstract

The invention discloses a children hand function rehabilitation system based on game interaction, relates to the technical field of rehabilitation training systems, and aims to provide personalized and dynamically adjusted rehabilitation treatment for children by combining deep learning, virtual reality, multi-modal perception feedback and cloud platform technologies. The system comprises a data acquisition module, an action analysis and recognition module, a game task generation module, a virtual reality module, a feedback and evaluation module and a data synchronization and remote monitoring module. By collecting the hand motion data of the child in real time and utilizing the deep learning model to analyze and generate the hand motion feature vector, the system can automatically adjust the difficulty of a game task according to the rehabilitation progress of the child, and stimulate the child to actively participate in treatment through a multi-mode feedback mechanism. And the reinforcement learning algorithm evaluates the task completion condition of the child in real time. The system has the advantages of high intelligence, individuation, real-time feedback and dynamic adjustment, and the rehabilitation treatment effect and treatment enthusiasm of children can be remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of rehabilitation training systems, and in particular to a children's hand function rehabilitation system based on game interaction. Background Art

[0002] Hand function rehabilitation refers to the restoration of hand motor ability through various means and methods, especially for the treatment of children's hand function loss caused by brain injury, nerve damage or developmental disorders. The rehabilitation treatment of children's hand function is not only a process of physical recovery, but also involves multiple aspects such as motor control, sensory perception, and psychological cognition.

[0003] At present, traditional methods of rehabilitation of children's hand function mainly include physical therapy, occupational therapy, functional training, and equipment-assisted rehabilitation. These methods help children restore limb motor function through repeated training of hand movements, especially training in finger flexibility, palm coordination, and grasping ability. However, these traditional methods have many limitations:

[0004] Monotony and lack of interest: Traditional rehabilitation training is mostly repetitive exercises, which can easily become boring for children, leading to unsatisfactory treatment results and low participation and enthusiasm of children.

[0005] Lack of personalization: Many rehabilitation programs cannot be dynamically adjusted according to the child’s specific recovery progress and differences in hand function, making it difficult to meet the personalized rehabilitation needs of each child.

[0006] Lag in monitoring and evaluation: Traditional treatments often rely on manual recording and evaluation. Real-time feedback and treatment adjustments during the rehabilitation process are difficult, resulting in an inability to respond to children's rehabilitation progress in a timely manner.

[0007] Backward technical means: Existing rehabilitation equipment is often mechanized, lacking intelligent means and adaptive adjustment capabilities during the treatment process, and cannot accurately match children's real-time performance and needs.

[0008] In recent years, with the development of technology, more and more studies have begun to try to introduce new technologies such as game interaction, virtual reality (VR) and artificial intelligence (AI) into the field of children's hand function rehabilitation. Game-based rehabilitation methods combine rehabilitation tasks with game situations to increase children's sense of participation and the fun of treatment, thereby improving children's rehabilitation effects. Specific features include:

[0009] Improve children's participation: Game interaction can effectively stimulate children's interest and enthusiasm for treatment, making the treatment process no longer boring and increasing children's training motivation.

[0010] Dynamic adjustment of task difficulty: Through the feedback mechanism in the game, the difficulty of the task can be adjusted in real time according to the child’s actual performance, ensuring that the task is both challenging and can be completed within the child’s ability, thereby optimizing the treatment effect.

[0011] Data-driven personalized treatment: By collecting children’s movement data in real time (such as finger activity, hand coordination, etc.), game tasks can be personalized according to the children’s rehabilitation progress, avoiding the “one-size-fits-all” problem in traditional treatment methods.

[0012] Multimodal feedback mechanism: With the help of multimodal technologies such as virtual reality, tactile feedback, and visual feedback, rehabilitation therapy not only relies on visual stimulation, but can also further enhance children's sense of participation and treatment experience through sensory input such as touch and hearing.

[0013] Although the game-based interactive rehabilitation method has many advantages in theory, there are still some problems in the application of related technologies:

[0014] Lack of comprehensive technology integration: Existing technologies are often limited to the application of a single technology, such as using virtual reality for hand rehabilitation training, or using artificial intelligence for data analysis, but lack a complete system that combines data collection, game task generation, real-time feedback and personalized treatment.

[0015] Neglect of individual differences among children: Currently, most game-based rehabilitation systems do not fully consider individual differences among children, including factors such as age, specific problems with hand function, and emotional responses. This makes it impossible to dynamically adjust the treatment plan based on the child’s rehabilitation progress and actual situation.

[0016] Insufficient real-time evaluation and feedback: Although games can provide feedback, this feedback is often based on the results of children completing tasks. There is a lack of real-time evaluation of details such as hand movement accuracy, strength, coordination, etc., and it is impossible to adjust treatment tasks in real time, making it difficult to meet more precise treatment needs.

[0017] Therefore, we urgently need to design a children's hand function rehabilitation system based on game interaction to solve the above problems. Summary of the invention

[0018] The purpose of the present invention is to provide a children's hand function rehabilitation system based on game interaction in view of the shortcomings of the prior art, so as to solve the problems raised in the background technology.

[0019] To achieve the above object, the present invention provides the following technical solutions:

[0020] A children's hand function rehabilitation system based on game interaction, the system includes the following modules:

[0021] Data acquisition module: used to collect children's hand motion data in real time through multiple sensor devices (such as cameras, inertial sensors, accelerometers, etc.) to generate a real-time motion data stream. The data acquisition module includes: a visual data acquisition unit, which collects images or depth image data of children's hands through cameras (such as RGB cameras, depth cameras); a motion data acquisition unit, which collects children's hand motion data (such as acceleration, angular velocity, etc.) through inertial sensors (such as accelerometers, gyroscopes, etc.).

[0022] Motion analysis and recognition module: used to input the children's hand movement data obtained by the data acquisition module into the deep learning model (such as convolutional neural network CNN, long short-term memory network LSTM, etc.), analyze and extract the feature vector X of the children's hand movement a , and generate the child's movement status to judge the progress of hand function recovery in real time, including:

[0023] X a =f(I v θ a ),

[0024] Among them, I v is the visual input data (image or video data) at time t, θ a is the parameter of the action analysis model, f(·) is the convolutional neural network (CNN) model, X a is the hand motion feature vector generated at time t.

[0025] Game tasks and virtual reality (VR) module: According to the action features X output by the analysis module a , generate personalized game tasks related to the progress of children's hand function recovery. Through virtual reality technology (VR), the game tasks are displayed in a virtual environment, and the difficulty of the tasks is dynamically adjusted through the task difficulty adjustment algorithm to adapt to the children's recovery progress. The specific adjustment algorithm is:

[0026] D t+1 =D t +α·(X a -X g ),

[0027] Among them, D t +1 is the difficulty of the next task, D t is the difficulty of the current task, α is the adjustment coefficient, X a is the current hand motion feature vector, X g is the target hand motion feature.

[0028] Feedback and evaluation module: used to generate real-time feedback based on the child's rehabilitation progress and provide personalized adjustment suggestions. Through deep learning and reinforcement learning algorithms, the recovery effect of children's hand function is evaluated. The evaluation formula is:

[0029]

[0030] Among them, Q(s t ,a t ) is the state s at time t t and action a t Q value, r t is the reward value, γ is the discount factor, is the maximum Q value of the next state.

[0031] Multimodal sensory feedback module: Based on real-time feedback, it provides multimodal feedback such as touch, vision, and hearing to enhance children's sense of participation in rehabilitation. The feedback intensity is adjusted by the following formula:

[0032] F total =α 1 ·F visual +β 1 ·F tactile +γ 1 ·F auditory ,

[0033] Among them, F total is the comprehensive feedback strength, F visual 、F tactile and F auditory are the strength of visual, tactile and auditory feedback, respectively, 1 ,β 1 , γ 1 is the feedback coefficient.

[0034] Data synchronization and remote monitoring module: The child's rehabilitation data is uploaded and synchronized to the doctor's and parents' monitoring systems in real time through the cloud platform, so that they can check the child's rehabilitation progress at any time and adjust the rehabilitation plan according to the data. The data synchronization formula is:

[0035]

[0036] Among them, D cloud To synchronize data in the cloud, D i is the data of the i-th device, and N is the number of devices.

[0037] As a preferred technical solution of the present invention, the system obtains children's hand motion data through real-time data acquisition and sensor data fusion. The acquisition process includes: using an RGB camera or a depth camera (such as Intel RealSense, Kinect, etc.) to capture children's hand image data, and extracting hand motion features through a convolutional neural network (CNN); using an inertial sensor (IMU, including an accelerometer and a gyroscope) to collect dynamic data such as motion acceleration and angular velocity of children's hands, and using LSTM to model the time series data for prediction and analysis.

[0038] As a preferred technical solution of the present invention, the task objectives in the virtual reality (VR) game interaction module are adjusted according to the progress of the child's movement recovery, and personalized game tasks are generated through the following algorithm formula:

[0039] T target =T base +λ 2 ·(S t -S target ),

[0040] Among them, T target is the target task, T base is the basic task, 2 is the adjustment factor, S t Score the current recovery, S target Score the target recovery.

[0041] As a preferred technical solution of the present invention, the parent and doctor monitoring system synchronizes data in real time through a cloud platform, so that parents and doctors can view the child's rehabilitation progress and adjust the treatment plan according to the following formula:

[0042] P t+1 =P t +λ 3 ·(S t+1 -S t ),

[0043] Among them, P t +1 for the next stage of treatment planning, P t is the current treatment plan, λ 3 is the adjustment factor, S t +1 and S t Respectively, they are the recovery scores for the current moment and the next moment.

[0044] As a preferred technical solution of the present invention, the system stores and shares cloud data through the following formula through a data synchronization and sharing mechanism:

[0045]

[0046] Among them, D cloud To synchronize data in the cloud, D i is the data of the i-th device, and N is the number of devices.

[0047] As a preferred technical solution of the present invention, the virtual reality (VR) game interaction module adjusts the task goal according to the progress of the child's movement recovery through a dynamic task adjustment algorithm, and updates the setting of the task goal using the following formula:

[0048] D t+1 =D t +β 2 ·(X a -X g ),

[0049] Among them, D t +1 is the difficulty of the next task, D t is the difficulty of the current task, β 2 is the adjustment coefficient, X a is the current hand motion feature vector, X g is the target action feature.

[0050] As a preferred technical solution of the present invention, the sensory feedback system includes visual feedback, tactile feedback, and auditory feedback, which together enhance the child's sense of participation and rehabilitation effect through the following formula:

[0051] F total =α 1 ·F visual +β 1 ·F tactile +γ 1 ·F auditory ,

[0052] Among them, F total is the comprehensive feedback strength, F visual 、F tactile and F auditory are the visual, tactile and auditory feedback strengths, respectively, 1 ,β 1 ,γ 1 is the feedback coefficient.

[0053] As a preferred technical solution of the present invention, the action recognition module optimizes the recognition performance of hand actions through a multimodal data fusion algorithm combining a convolutional neural network (CNN) and a long short-term memory network (LSTM), and optimizes feature extraction through the following formula:

[0054]

[0055] Among them, L is the loss function, y i is the actual output, is the predicted output, θ j is the model parameter, λ 4 is the regularization coefficient, N is the number of samples, and M is the number of parameters.

[0056] As a preferred technical solution of the present invention, the parent and doctor monitoring system provides real-time data sharing and monitoring through a cloud platform, allowing doctors to remotely view and adjust treatment plans.

[0057] As a preferred technical solution of the present invention, the virtual reality (VR) game task generation process is optimized based on a deep learning algorithm, and the task difficulty is adjusted by continuously feeding back the children's rehabilitation effects to ensure maximum therapeutic effects.

[0058] Compared with the prior art, the present invention has the following beneficial effects:

[0059] 1. The game-interactive children's hand function rehabilitation system of the present invention combines game interaction with treatment tasks, which not only effectively stimulates children's interest and enthusiasm in treatment, avoids the dullness of traditional rehabilitation methods, but also enhances the fun and interactivity of treatment. The multimodal perceptual feedback mechanism (such as tactile, visual, and auditory feedback) provides real-time rewards and feedback, further enhancing children's sense of participation and treatment motivation, thereby improving the overall treatment effect.

[0060] 2. The system of the present invention can evaluate the rehabilitation progress of each child in real time based on the hand movement data, and adjust the difficulty and content of the treatment tasks according to individual differences. Through the analysis of the movement data by the deep learning model, the recovery of the child's hand function can be accurately captured, and the treatment plan can be dynamically adjusted to ensure that the challenge of the task matches the child's recovery ability, avoiding the "one-size-fits-all" problem in traditional treatment methods and providing truly personalized treatment.

[0061] 3. The present invention realizes real-time monitoring of the child's rehabilitation process through an intelligent feedback mechanism and cloud platform data synchronization function. Doctors and parents can obtain children's rehabilitation data at any time and make adjustments according to the progress of treatment to ensure the optimization of the treatment plan. This remote monitoring not only improves the accuracy of treatment, but also improves the control and participation of parents and doctors in children's rehabilitation, ensuring the continuity and stability of rehabilitation treatment. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 This is a system block diagram of the children's hand function rehabilitation system based on game interaction proposed by the present invention. DETAILED DESCRIPTION

[0063] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] The following is combined with Figure 1 With reference to the accompanying drawings and a number of embodiments, specific embodiments of the present invention are described in detail.

[0065] 1. System overall framework:

[0066] The present invention provides a children's hand function rehabilitation system based on game interaction. The overall framework of the system consists of multiple modules, including a data acquisition module, a motion analysis and recognition module, a game task and virtual reality (VR) module, a feedback and evaluation module, a multimodal perception feedback module, and a data synchronization and remote monitoring module. Each module provides a personalized and progressive rehabilitation treatment plan through real-time collaboration.

[0067] 2. Data acquisition module:

[0068] The data acquisition module collects children's hand movement data in real time through a variety of sensor devices. These sensors include visual data acquisition devices: RGB cameras or depth cameras (such as Intel RealSense, Kinect, etc.) are used to collect images or depth images of children's hands. Image data is used to capture the movements and postures of children's hands.

[0069] Motion data acquisition equipment: captures dynamic data such as hand movement acceleration and angular velocity through inertial sensors (such as accelerometers, gyroscopes, etc.). These data are then transmitted to the motion analysis and recognition module for processing.

[0070] 3. Motion analysis and recognition module:

[0071] The core of the motion analysis and recognition module is the deep learning model. This module analyzes and recognizes children's hand movements through the following steps:

[0072] Visual data input: Image data obtained through RGB camera or depth camera v The image is fed into a convolutional neural network (CNN) for feature extraction, which extracts the key points, contours, and motion trajectories of the hand from the image.

[0073] Motion data input: The acceleration, angular velocity and other data collected by the inertial sensor (IMU) are input into the long short-term memory network (LSTM) for time series analysis to capture the movement patterns of children’s hands.

[0074] Feature vector generation: After fusing the feature data extracted by CNN and LSTM, the feature vector X of the child’s hand movement is generated. a , obtained by the following formula:

[0075] X a =f(I v θ a ),

[0076] Among them, I v is the visual input data, θ a are the parameters of the model, f(·) is the convolutional neural network (CNN) function, and X a is the generated hand motion feature vector.

[0077] The feature vector X a It contains various details of children's hand movements, such as grip strength, movement range and movement speed. This information will be further used to generate game tasks and adjust treatment progress.

[0078] 4. Game tasks and virtual reality (VR) module:

[0079] The hand motion feature vector X generated by the motion analysis and recognition module a The game task and virtual reality module generates personalized game tasks and presents them to children through virtual reality technology. During the task generation process, the system dynamically adjusts the difficulty of the task according to the child's movement status (such as the progress of hand function recovery) to ensure that the task is challenging but does not exceed the child's ability. The task adjustment formula is:

[0080] D t+1 =D t +α·(X a -X g ),

[0081] Among them, D t is the current task difficulty, D t +1 is the difficulty of the next task, α is the adjustment coefficient, X a is the current hand motion feature vector, X g is the target hand motion feature vector.

[0082] Through this method, the system can gradually increase the difficulty of tasks according to the child's rehabilitation progress, thereby achieving progressive treatment and avoiding boredom and lack of challenge in treatment.

[0083] 5. Feedback and evaluation module:

[0084] The feedback and evaluation module is used to evaluate the child's performance in the process of completing the task in real time, and dynamically adjust the task and treatment plan according to the child's recovery. This module uses a reinforcement learning algorithm to evaluate the progress of hand function recovery in real time and provide feedback. The treatment progress is evaluated using the Q learning algorithm, and the evaluation formula is:

[0085]

[0086] Among them, Q(s t ,a t ) is state s t and action a t Q value, r t is the reward value, γ is the discount factor, is the maximum Q value of the next state.

[0087] In this way, the system is able to adjust the difficulty of the therapy task in real time to better suit the child's hand recovery.

[0088] 6. Multimodal perception feedback module:

[0089] In order to improve children's participation in treatment, the multimodal sensory feedback module provides multiple forms of feedback, such as vision, touch and hearing. These feedbacks can prompt children's task completion in real time and encourage them to continue to participate in treatment. The feedback intensity is adjusted by the following formula:

[0090] F total =α 1 ·F visual +β 1 ·F tactile +γ 1 ·F auditory ,

[0091] Among them, F total is the comprehensive feedback strength, F v isual, F tactile and F auditory are the visual, tactile and auditory feedback strengths, respectively, 1 , β 1 , γ 1 is the feedback coefficient. Visual feedback: Through VR equipment, the task progress and children’s hand movements are displayed in real time to help children clearly understand whether their movements are correct.

[0092] Haptic feedback: Provide tactile feedback to children when they complete a task, using a vibrating glove or other tactile feedback device.

[0093] Auditory feedback: Use sound effects or voice prompts to encourage children to be more focused and active during treatment.

[0094] 7.Data synchronization and remote monitoring module:

[0095] The data synchronization and remote monitoring module realizes real-time synchronization and remote monitoring of rehabilitation data. Through the cloud platform, doctors and parents can view the child's rehabilitation progress in real time and adjust the treatment plan based on the data. The specific data synchronization formula is:

[0096]

[0097] Among them, D cloud To synchronize data in the cloud, D i is the data of the i-th device, and N is the number of devices. Doctors can adjust the child’s treatment plan based on the real-time feedback provided by the system and provide remote guidance.

[0098] The present invention provides a children's hand function rehabilitation system based on game interaction. Through the collaborative work of multiple modules, the system can provide children with personalized rehabilitation plans, adjust the difficulty of tasks in real time, and provide multi-modal feedback. This embodiment describes in detail how to use the system to perform hand rehabilitation training for children with hand dysfunction.

[0099] Example 1: Virtual reality-based rehabilitation treatment process for children's hand function;

[0100] Step 1: Data Collection:

[0101] Visual data collection: Before starting treatment, the system uses an RGB camera or a depth camera to collect preliminary images of the child's hand. The data includes the posture, position, bending degree of fingers, palm movement, etc. of the child's hand. Image data I v It will be used for subsequent feature extraction.

[0102] Sports data collection:

[0103] In addition, the system also collects the acceleration and angular velocity of children's hands in real time through inertial sensors (such as accelerometers and gyroscopes). These data can record the dynamic information of children's hand movement speed, strength, etc. The motion data and visual data together constitute the children's motion data stream.

[0104] Step 2: Motion analysis and recognition:

[0105] Feature extraction: After data collection, the system will process the visual data through a convolutional neural network (CNN) to extract key features of the child's hand, such as the movement trajectory of the fingers, the movement pattern of the hand, etc. At the same time, the system also uses a long short-term memory network (LSTM) to perform time series modeling on the motion data to capture the time series characteristics of the child's hand movements.

[0106] Generate action feature vector: Input the processed visual data and motion data into the deep learning model to generate a hand action feature vector X a , represents the current state of the child’s hand. This feature vector contains various parameters of hand movement, such as finger independence, coordination, and movement stability. a It will provide a basis for the generation and difficulty adjustment of subsequent game tasks.

[0107] Step 3: Generate personalized game tasks:

[0108] Based on the child’s current hand motion feature vector X a , the system will generate personalized rehabilitation game tasks. For example, if the child's finger independence is poor, the system may design some tasks to train the independent movement of fingers, such as clicking virtual buttons of different colors, etc. If the child needs to improve grip strength and finger coordination, the system may design virtual object grasping tasks, etc.

[0109] Task Difficulty Adjustment: The difficulty of the task generation game tasks will be adjusted in real time according to the child's rehabilitation progress. Through the following formula:

[0110] D t+1 =D t +α·(X a -X g ),

[0111] Among them, D t is the difficulty of the current task, D t+1 is the difficulty of the next task, α is the adjustment coefficient, X a is the current hand motion feature, X g is the target hand motion feature.

[0112] This formula ensures that the difficulty of the task can be gradually increased or adjusted as the child's recovery progresses, avoiding tasks that are too difficult or too easy.

[0113] Step 4: Feedback and Evaluation:

[0114] Real-time feedback: When children are performing tasks, the system will provide real-time feedback through a multimodal feedback system to encourage them to complete the task. For example, if a child completes a task, the system will remind the child of the task completion through tactile feedback (such as vibration), visual feedback (such as task completion animation) and auditory feedback (such as prompt sound), thereby enhancing the child's rehabilitation enthusiasm.

[0115] Evaluation and adjustment: At the same time, the system will use the reinforcement learning algorithm to evaluate the child's task completion in real time and adjust the difficulty of subsequent tasks based on the evaluation results. The evaluation formula is:

[0116]

[0117] Among them, Q(s t ,a t ) is the state s at time t t and action a t Q value, r t is the reward value, γ is the discount factor, is the maximum Q value of the next state.

[0118] The system dynamically adjusts the difficulty of the task based on this value, so that children are always receiving treatment at an appropriately challenging level.

[0119] Step 5: Data synchronization and remote monitoring;

[0120] Data synchronization: All collected motion data, task completion status, and rehabilitation assessment results will be synchronized in real time through the cloud platform. Doctors and parents can view the child's rehabilitation progress at any time through the cloud platform and adjust the treatment plan. Remote monitoring and adjustment Doctors and parents can view the child's real-time rehabilitation data (for example, the accuracy of hand movements, task completion status, etc.) through the monitoring interface and make necessary adjustments based on the data to optimize the rehabilitation plan.

[0121] Embodiment 2: Data collection;

[0122] Step 1: Visual Data Collection,A 10-year-old child with hand coordination disorder was given rehabilitation treatment. First, the child’s hand image was captured in real time using a depth camera. v It is used to extract children’s hand movement trajectory and movement accuracy.

[0123] Motion data collection In addition, the “inertial measurement unit (IMU)” is used to collect the acceleration and angular velocity data of children’s hand movements, generate time series data, and perform time series modeling.

[0124] Step 2: Action analysis and recognition;

[0125] Through the combination of CNN and LSTM models, the image data and motion data are processed and extracted to generate the feature vector X of the current hand movement. a , which indicates a child’s hand coordination and range of motion.

[0126] Step 3: Task generation and difficulty adjustment;

[0127] X-based a , generating personalized virtual game tasks (such as grabbing a virtual object and moving it to a target location). As the child's hand coordination improves, the system gradually increases the difficulty of the task, using the formula:

[0128] Dt+1 =D t +α·(X a -X g ),

[0129] Ensure children are continually challenged without feeling overly challenged.

[0130] Step 4: Feedback and evaluation;

[0131] The system provides multimodal feedback in real time based on task completion. Each time a task is completed, the system rewards the child through tactile and visual feedback. The system also dynamically adjusts the task difficulty through a Q-learning algorithm to ensure that the task is always appropriate for the child's recovery progress.

[0132] Step 5: Data synchronization and remote monitoring;

[0133] The cloud platform synchronizes the child's rehabilitation data in real time, allowing parents and doctors to remotely monitor the treatment progress. If the doctor believes that the treatment plan needs to be adjusted, the task content or difficulty can be modified directly on the cloud platform to ensure that the child receives appropriate treatment.

[0134] The contents not described in detail in this specification belong to the prior art known to professional and technical personnel in the field. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A children's hand function rehabilitation system based on game interaction, characterized in that: The system includes the following modules: Data acquisition module: used to collect the hand motion data of the child in real time through multiple sensor devices to generate a real-time motion data stream, the data acquisition module includes: a visual data acquisition unit, which collects the image or depth image data of the child's hand through a camera; A motion data collection unit collects motion data of children’s hands through inertial sensors; Motion analysis and recognition module: used to input the children's hand movement data obtained by the data acquisition module into the deep learning model, analyze and extract the feature vector X of the children's hand movements a , and generate the child's movement status to judge the progress of hand function recovery in real time, including: X a =f(I v ;θ a ), Among them, I v is the visual input data (image or video data) at time t, θ a is the parameter of the action analysis model, f(·) is the convolutional neural network model, X a is the current hand motion feature vector; Game tasks and virtual reality module: according to the action features X output by the analysis module a , generate personalized game tasks related to the progress of children's hand function recovery, display the game tasks in a virtual environment through virtual reality technology, and dynamically adjust the difficulty of the tasks through the task difficulty adjustment algorithm to adapt to the children's recovery process. The specific adjustment algorithm is: D t+1 =D t +α·(X a -X g ), Among them, D t +1 is the difficulty of the next task, D t is the difficulty of the current task, α is the adjustment coefficient, X a is the current hand motion feature vector, X g is the target hand motion feature; Feedback and evaluation module: used to generate real-time feedback based on the child's rehabilitation progress and provide personalized adjustment suggestions. Through deep learning and reinforcement learning algorithms, it evaluates the recovery effect of children's hand function. The evaluation formula is: Among them, Q(s t ,a t ) is the state s at time t t and action a t Q value, r t is the reward value, γ is the discount factor, is the maximum Q value of the next state; Multimodal sensory feedback module: Based on real-time feedback, it provides tactile, visual, and auditory multimodal feedback to enhance children's sense of participation in rehabilitation. The feedback intensity is adjusted by the following formula: F total =α1·F visual +β1·F tactile +γ1·F auditory , Among them, F total is the comprehensive feedback strength, F visual 、F tactile and F auditory are the intensities of visual, tactile and auditory feedback, respectively, and α1, β1, and γ1 are feedback coefficients; Data synchronization and remote monitoring module: The child's rehabilitation data is uploaded and synchronized to the doctor's and parents' monitoring systems in real time through the cloud platform, so that they can check the child's rehabilitation progress at any time and adjust the rehabilitation plan according to the data. The data synchronization formula is: Among them, D cloud To synchronize data in the cloud, D i is the data of the i-th device, and N is the number of devices.

2. The game-interactive children's hand function rehabilitation system according to claim 1, characterized in that: The system obtains children's hand motion data through real-time data acquisition and sensor data fusion. The acquisition process includes: using an RGB camera or a depth camera to capture children's hand image data, and extracting hand motion features through a convolutional neural network; using an inertial sensor to collect children's hand motion acceleration and angular velocity dynamic data, and using LSTM to model the time series data for prediction and analysis.

3. The game-interactive children's hand function rehabilitation system according to claim 1, characterized in that: The task objectives in the virtual reality game interaction module are adjusted according to the progress of the child's movement recovery, and personalized game tasks are generated through the following algorithm formula: T target =T base +λ2·(S t -S target ), Among them, T target is the target task, T base is the basic task, λ2 is the adjustment factor, S t Score the current recovery, S target Score the target recovery.

4. The game-interactive children's hand function rehabilitation system according to claim 1, characterized in that: The parent and doctor monitoring system synchronizes data in real time through the cloud platform. Parents and doctors can view the child's recovery progress and adjust the treatment plan according to the following formula: P t+1 =P t +λ3·(S t+1 -S t ), Among them, P t +1 for the next stage of treatment planning, P t is the current treatment plan, λ3 is the adjustment factor, S t +1 and S t Respectively, they are the recovery scores for the current moment and the next moment.

5. The game-interactive children's hand function rehabilitation system according to claim 1, characterized in that: The system uses a data synchronization and sharing mechanism to store and share cloud data using the following formula: Among them, D cloud To synchronize data in the cloud, D i is the data of the i-th device, and N is the number of devices.

6. The game-interactive children's hand function rehabilitation system according to claim 1, characterized in that: The sensory feedback system includes visual feedback, tactile feedback, and auditory feedback, which together enhance children's sense of participation and rehabilitation effects through the following formula: F total =α1·F visual +β1·F tactile +γ1·F auditory , Among them, F total is the comprehensive feedback strength, F visual 、F tactile and F auditory are the visual, tactile and auditory feedback intensities respectively, and α1, β1, γ1 are the feedback coefficients.

7. The game-interactive children's hand function rehabilitation system according to claim 1, characterized in that: The action recognition module optimizes the recognition performance of hand actions through a multimodal data fusion algorithm combining a convolutional neural network and a long short-term memory network, and optimizes feature extraction through the following formula: Among them, L is the loss function, y i is the actual output, is the predicted output, θ j is the model parameter, λ4 is the regularization coefficient, N is the number of samples, and M is the number of parameters.

8. The game-interactive children's hand function rehabilitation system according to claim 1, characterized in that: The parent and doctor monitoring system provides real-time data sharing and monitoring through a cloud platform, allowing doctors to remotely view and adjust treatment plans.

9. The game-interactive children's hand function rehabilitation system according to claim 1, characterized in that: The virtual reality game task generation process is optimized based on a deep learning algorithm, and the task difficulty is adjusted by continuously providing feedback on the children's rehabilitation effects to ensure maximum treatment effects.

Images