Limb rehabilitation training method and system based on digital twinborn and AI technologies

Abstract

The invention provides a limb rehabilitation training method and system based on digital twinning and AI technology, and the system processes the motion data of a patient in real time based on the digital twinning and AI technology, and controls a virtual digital person to carry out joint rehabilitation training prompt in real time based on the motion data. Real-time monitoring can be carried out in the limb rehabilitation training process of a patient, a personalized rehabilitation report is generated in time through the AI technology, and then the rehabilitation period of the patient is shortened.

Description

Technical Field

[0001] The present invention relates to the field of limb rehabilitation training, and in particular to a limb rehabilitation training method and system based on digital twin and AI technologies. Background Art

[0002] Limb rehabilitation training refers to helping patients restore the functions of parts such as shoulder joints, elbow joints, hip joints, knee joints, and ankle joints through specific training and auxiliary technologies, so as to improve their mobility and quality of life. During the process of limb rehabilitation training for patients, doctors are unable to adjust the training actions of patients at any time, nor can they timely obtain the rehabilitation reports of patients, which results in a longer rehabilitation period for patients.

[0003] Therefore, how to perform real-time monitoring during the limb rehabilitation training process of patients and timely generate rehabilitation reports has become an urgent problem to be solved. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a limb rehabilitation training method and system based on digital twin and AI technologies. Based on digital twin and AI technologies, the motion data of patients is processed in real time, and based on this motion data, the virtual digital human is controlled to perform joint rehabilitation training prompts in real time, so as to be able to perform real-time monitoring during the limb rehabilitation training process of patients, and use AI technology to timely generate personalized rehabilitation reports, thereby shortening the rehabilitation period of patients.

[0005] In a first aspect, an embodiment of this application provides a limb rehabilitation training method based on digital twin and AI technologies, and the method includes:

[0006] In response to the limb rehabilitation training request of the patient, play a standard training video of a preset training joint corresponding to the current training limb of the patient on the client of the patient, so that the patient performs training according to the standard training video; and obtain the motion data of the preset training joint in the current time period in real time;

[0007] Using digital twin technology, generate the motion trajectory of the virtual digital human from the previous moment to the current moment displayed on the client of the patient according to the motion data in the current time period, and control the virtual digital human according to the motion trajectory;

[0008] Compare the virtual digital human with the demonstrator in the simultaneously played standard training video according to the preset rehabilitation training action parameters corresponding to the preset training joint;

[0009] Perform real-time prompts for joint rehabilitation training on the patient according to the comparison result;

[0010] After the training is completed, a rehabilitation report for the patient is generated according to the actual training index parameters of the patient during the limb rehabilitation training process and the large language model obtained by using the fine-tuning technology; the actual training index parameters are determined based on all the comparison results during the training process.

[0011] In a possible implementation manner, generating the motion trajectory of the virtual digital human from the previous moment to the current moment displayed on the client of the patient according to the motion data in the current time period includes:

[0012] Determine the joint angle of the preset training joint at the current moment according to the motion data in the current time period;

[0013] Generate the motion trajectory of the virtual digital human from the previous moment to the current moment according to the motion data in the current time period and the joint angle at the current moment.

[0014] In a possible implementation manner, determining the joint angle of the preset training joint at the current moment according to the motion data in the current time period includes:

[0015] Calculate the first joint angle of the preset training joint at the current moment according to the motion data at the current moment;

[0016] Input the motion data in the current time period into the joint angle estimation model to obtain the second joint angle of the preset training joint at the current moment;

[0017] Calculate the target joint angle of the preset training joint at the current moment according to the first joint angle and the second joint angle.

[0018] In a possible implementation manner, the motion data includes the motion data of each preset body part in the preset body part pair corresponding to the preset training joint; calculating the first joint angle of the preset training joint at the current moment according to the motion data at the current moment includes:

[0019] For each preset body part, calculate the pose vector of the preset body part at the current moment according to the motion data of the preset body part at the current moment;

[0020] Calculate the first joint angle of the preset training joint at the current moment according to the pose vectors of all the preset body parts at the current moment.

[0021] In a possible implementation manner, calculating the first joint angle of the preset training joint at the current moment according to the pose vectors of all the preset body parts at the current moment includes:

[0022] Substitute the pose vectors of all the preset body parts at the current moment into the following formula to obtain the first joint angle of the preset training joint at the current moment:

[0023]

[0024] where θ init (t) is the first joint angle of the preset training joint at the current moment, is the pose vector of the first preset body part in the alignment of the preset body parts at the current moment, is the pose vector of the second preset body part in the alignment of the preset body parts at the current moment, is the modulus of the pose vector of the first preset body part in the alignment of the preset body parts at the current moment ; is the modulus of the pose vector of the second preset body part in the alignment of the preset body parts at the current moment ;

[0025] In a possible implementation manner, calculating the target joint angle of the preset training joint at the current moment according to the first joint angle and the second joint angle includes:

[0026] Substitute the first joint angle and the second joint angle into the following formula to obtain the target joint angle of the preset training joint at the current moment;

[0027] g = σ(W g ·[θ init (t), θ t + b g );

[0028] θ final (t) = g·θ init (t) + (1 - g)·θ t ;

[0029] where g is the gating mechanism, σ is the activation function, W g is the weight matrix, θ init (t) is the first joint angle at the current moment t, θ t is the second joint angle at the current moment t, b g is the bias, and θ final (t) is the target joint angle at the current moment t.

[0030] In a possible implementation manner, the method further includes:

[0031] Generate a prompt word according to the actual training index parameter, the preset training index parameter, and the preset joint rehabilitation report generation rule;

[0032] Input the prompt into the large language model to obtain the rehabilitation report of the patient.

[0033] In a second aspect, an embodiment of the present application further provides a limb rehabilitation training system based on digital twin and AI technologies. The system includes:

[0034] A response module, configured to respond to a limb rehabilitation training request of a patient, play a standard training video of a preset training joint corresponding to the patient's current training limb on the patient's client, so that the patient trains according to the standard training video; and obtain motion data of the preset training joint in the current time period in real time;

[0035] A generation control module, configured to use digital twin technology to generate a motion trajectory of a virtual digital human from the previous moment to the current moment displayed on the patient's client according to the motion data in the current time period, and control the virtual digital human according to the motion trajectory;

[0036] A comparison module, configured to compare the virtual digital human with a demonstrator in the simultaneously played standard training video according to preset rehabilitation training action parameters corresponding to the preset training joint;

[0037] A prompt module, configured to perform real-time prompts for the patient's joint rehabilitation training according to the comparison result;

[0038] A generation module, configured to generate a rehabilitation report of the patient according to the actual training index parameters of the patient during the limb rehabilitation training process and a large language model obtained by using the fine-tuning technology after the training is completed; the actual training index parameters are determined based on all the comparison results during the training process.

[0039] In a third aspect, an embodiment of the present application further provides an electronic device, including: a processor, a storage medium, and a bus. The storage medium stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to perform the steps of the limb rehabilitation training method based on digital twin and AI technologies as described in any item of the first aspect.

[0040] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it performs the steps of the limb rehabilitation training method based on digital twin and AI technologies as described in any item of the first aspect.

[0041] The embodiments of the present application provide a limb rehabilitation training method and system based on digital twin and AI technologies. The method includes: in response to a patient's limb rehabilitation training request, playing a standard training video corresponding to the preset training joints of the patient's current training limb on the patient's client, so that the patient can train according to the standard training video; and obtaining the motion data of the preset training joints in the current time period in real time; using digital twin technology, generating the motion trajectory of the virtual digital human from the previous moment to the current moment displayed on the patient's client according to the motion data in the current time period, and controlling the virtual digital human according to the motion trajectory; comparing the virtual digital human with the demonstrators in the simultaneously played standard training video according to the preset rehabilitation training action parameters corresponding to the preset training joints; giving real-time prompts for the patient's joint rehabilitation training according to the comparison result; after the training is completed, generating a rehabilitation report for the patient according to the actual training index parameters during the limb rehabilitation training process of the patient and the large language model obtained by using the fine-tuning technology; the actual training index parameters are determined based on all the comparison results during the training process. Based on digital twin and AI technologies, the motion data of the patient is processed in real time, and real-time prompts for joint rehabilitation training are controlled based on the motion data, so as to be able to monitor the patient in real time during the limb rehabilitation training process, and use AI technology to generate personalized rehabilitation reports in a timely manner, thereby shortening the patient's rehabilitation cycle. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0043] Figure 1 FIG. shows a schematic flow chart of a limb rehabilitation training method based on digital twin and AI technologies provided by an embodiment of the present application;

[0044] Figure 2 FIG. shows a flowchart for determining joint angles provided by an embodiment of the present application;

[0045] Figure 3 FIG. shows a schematic diagram of a limb rehabilitation training system based on digital twin and AI technologies provided by an embodiment of the present application;

[0046] Figure 4 FIG. shows a schematic structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0047] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this application. It should be understood that the accompanying drawings in this application are only for the purposes of illustration and description, and are not used to limit the protection scope of this application. Additionally, it should be understood that the schematic drawings are not drawn to actual scale. The flowcharts used in this application illustrate the operations implemented according to some embodiments of this application. It should be understood that the operations in the flowchart may not be implemented in sequence, and steps without a logical context relationship may be reversed or implemented simultaneously. Moreover, those skilled in the art can add one or more other operations to the flowchart or remove one or more operations from the flowchart under the guidance of the content of this application.

[0048] Furthermore, the described embodiments are only a part of the embodiments of this application, rather than all of the embodiments. The components of the embodiments of this application usually described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of this application claimed, but merely represents the selected embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative efforts belong to the protection scope of this application.

[0049] To enable those skilled in the art to use the content of this application, the following implementation manners are given in combination with a specific application scenario, the "field of limb rehabilitation training". For those skilled in the art, the general principles defined here can be applied to other embodiments and application scenarios without departing from the spirit and scope of this application. Although this application is mainly described around the "field of limb rehabilitation training", it should be understood that this is only an exemplary embodiment.

[0050] It should be noted that the term "including" will be used in the embodiments of this application to indicate the existence of the features stated thereafter, but does not exclude the addition of other features.

[0051] The following provides a detailed description of a limb rehabilitation training method based on digital twin and AI technologies provided by the embodiments of this application.

[0052] Refer to Figure 1 As shown, it is a schematic flowchart of a limb rehabilitation training method based on digital twin and AI technologies provided by the embodiments of this application. The following explains each step of the embodiments of this application exemplarily:

[0053] S101. In response to the patient's request for limb rehabilitation training, play a standard training video of the preset training joint corresponding to the patient's current training limb on the patient's client, so that the patient can train according to the standard training video; and obtain the motion data of the preset training joint in the current time period in real time.

[0054] In the embodiment of the present application, before responding to the patient's request for limb rehabilitation training, obtain the patient's name, age, limb to be trained (including the current training limb), joint for restoring the joint range of motion of the limb to be trained (i.e., the preset training joint corresponding to the limb to be trained, including the preset training joint corresponding to the current training limb), historical rehabilitation situation, preset training index parameters, etc. input by the doctor on the corresponding client (tablet or mobile phone, etc.). The standard training video of the preset training joint played can be selected by the patient or formulated by the doctor for the patient.

[0055] In addition, standard training videos of each limb are pre-stored in the standard video library. There is at least one different rehabilitation training action between the rehabilitation training videos. The standard training video is a video of the standard rehabilitation training action recorded by the demonstrator.

[0056] The method further includes: before playing the standard training video of the preset training joint corresponding to the patient's current training limb on the patient's client, display a calibrated virtual digital human corresponding to the preset calibration pose to the patient's client, so that the patient can make self-adjustment according to the preset calibration pose.

[0057] S102. Using digital twin technology, generate the motion trajectory of the virtual digital human displayed on the patient's client from the previous moment to the current moment according to the motion data in the current time period, and control the virtual digital human according to the motion trajectory.

[0058] Step 1. Determine the joint angle of the preset training joint at the current moment according to the motion data in the current time period.

[0059] In the embodiment of the present application, the joint angle refers to the included angle formed between adjacent bones during the movement of the joint. The motion data includes the motion data of each preset body part of the preset body part pair corresponding to the preset training joint. Different preset training joints correspond to different preset body part pairs, and each preset body part pair includes two preset body parts. The patient wears an IMU sensor (inertial measurement unit) on each preset body part of the preset body part pair corresponding to the preset training joint in advance to obtain the motion data of the preset body part. The motion data includes acceleration data, angular velocity data, magnetometer data, etc.

[0060] Among them, the joints can include the shoulder joint, elbow joint, hip joint, knee joint and ankle joint; the preset body parts in the preset body part pair corresponding to the left shoulder joint include the left upper arm and the middle position of the front chest; the preset body parts in the preset body part pair corresponding to the right shoulder joint include the right upper arm and the middle position of the front chest; the preset body parts in the preset body part pair corresponding to the left elbow joint include the left upper arm and the left lower arm; the preset body parts in the preset body part pair corresponding to the right elbow joint include the right upper arm and the right lower arm; the preset body parts in the preset body part pair corresponding to the hip joint include the middle position of the waist, the thighs (left thigh and / or right thigh); the preset body parts in the preset body part pair corresponding to the left knee joint include the left thigh and the left lower leg; the preset body parts in the preset body part pair corresponding to the right knee joint include the right thigh and the right lower leg; the preset body parts in the preset body part pair corresponding to the left ankle joint include the left lower leg and the left instep; the preset body parts in the preset body part pair corresponding to the right ankle joint include the right lower leg and the right instep.

[0061] Here, different joints correspond to different rehabilitation training actions; specifically:

[0062] The rehabilitation training actions corresponding to the shoulder joint include the abduction action, elevation action and rotation action of the shoulder, helping the patient restore the flexibility and control ability of the shoulder joint. During the training process, the patient needs to perform the abduction training of the shoulder joint, that is, the arm is horizontally lifted outward from the side of the body to a position as high as the shoulder as much as possible, while keeping the shoulder relaxed and avoiding shrugging; in the elevation training, the patient needs to lift the arm forward to the position of the chest or the top of the head, and gradually increase the amplitude of lifting the arm; the rotation training requires the patient to perform the internal rotation and external rotation actions of the shoulder joint, for example, the forearm is fixed on the side of the body, and the palm rotates inward or outward.

[0063] The rehabilitation training of the elbow joint focuses on the control and coordination of the flexion and extension actions. During the training process, the patient needs to bend the forearm inward to reach the maximum flexion angle of the elbow, and then slowly extend it to the fully extended position, paying attention to controlling the strength to avoid muscle tension or pain caused by too fast or too violent actions.

[0064] The rehabilitation training actions of the hip joint include flexion and extension actions and external rotation actions, etc., to restore the stability and flexibility of the hip. In the flexion and extension action, the patient needs to lift the thigh to the height of the hip and slowly lower it, simulating the flexion and extension of the hip joint in walking or squatting actions; the external rotation action requires the patient to keep the knees bent in a standing or sitting position and rotate the thigh outward by a small angle to train the internal rotation and external rotation ability of the hip joint.

[0065] The rehabilitation training movements of the knee joint include flexion and extension movements to restore the flexion and extension range and stability of the knee. The patient needs to perform the knee flexion movement, bend the lower leg backward as much as possible to the maximum flexion degree, and then slowly straighten the lower leg to a fully relaxed position. This training is similar to the movements of going up and down stairs or squatting and standing up, which helps to enhance the supporting ability of the muscles around the knee joint.

[0066] The rehabilitation training movements of the ankle joint include flexion, extension and rotation movements of the ankle joint to help restore the flexibility and coordination of the ankle. The patient needs to perform the movements of lifting the toes (dorsiflexion) and pressing down the toes (plantar flexion) to simulate the natural movement of the ankle joint during walking and gradually increase the range of motion of the ankle joint; the rotation training requires the patient to rotate the sole of the foot clockwise or counterclockwise around the ankle joint to train the multi-directional control ability of the ankle.

[0067] Refer to Figure 2 As shown, it is the flowchart for determining the joint angle provided by the embodiment of the present application; according to the motion data within the current time period, determine the joint angle of the preset training joint at the current moment, including:

[0068] S201. Calculate the first joint angle of the preset training joint at the current moment according to the motion data at the current moment.

[0069] i. For each preset body part, calculate the pose vector of the preset body part at the current moment according to the motion data of the preset body part at the current moment.

[0070] In the embodiment of the present application, substitute the motion data of the preset body part at the current moment into the following formula to obtain the pose vector of the preset body part at the current moment:

[0071]

[0072] R t = exp(ω(t)Δt);

[0073] Wherein, is the pose vector of the preset body part at the current moment, is the pose vector of the preset body part at time 0, R t is the pose change of the preset body part from time 0 to the current moment t, exp(·) is the matrix exponential operation, ω(t) is the angular velocity data in the motion data of the preset body part at the current moment t, and Δt is the time interval between adjacent time steps.

[0074] ii. Calculate the first joint angle of the preset training joint at the current moment according to the pose vectors of all preset body parts at the current moment.

[0075] In the embodiment of the present application, the pose vectors of all preset body parts at the current moment are substituted into the following formula to obtain the first joint angle of the preset training joint at the current moment:

[0076]

[0077] where θ init (t) is the first joint angle of the preset training joint at the current moment, is the pose vector of the first preset body part in the alignment of the preset body parts at the current moment, is the pose vector of the second preset body part in the alignment of the preset body parts at the current moment, is the modulus of the pose vector of the first preset body part in the alignment of the preset body parts at the current moment of, is the modulus of the pose vector of the second preset body part in the alignment of the preset body parts at the current moment of.

[0078] S202. Input the motion data within the current time period into the joint angle estimation model to obtain the second joint angle of the preset training joint at the current moment.

[0079] In the embodiment of the present application, the joint angle estimation model includes a convolutional feature extraction module, a long short-term memory network (LSTM) modeling module, and an attention module; the joint angle estimation model is trained according to motion sample data and the corresponding joint angles.

[0080] Here, inputting the motion data within the current time period into the joint angle estimation model to obtain the second joint angle of the preset training joint at the current moment includes:

[0081] i. Input the motion data within the current time period into the convolutional feature extraction module to obtain the convolutional features of the preset training joint at the current moment.

[0082] In the embodiment of the present application, the motion data within the current time period is substituted into the following formula to obtain the convolutional features of the preset training joint at the current moment:

[0083]

[0084] where y t is the convolutional feature of the preset training joint at the current moment, CNN is a convolutional neural network, K is the size of the convolutional kernel, W i is the convolutional kernel weight, x t-i is the feature vector of the motion data at the t-i moment, x t is the feature vector of the motion data at the current moment t.

[0085] ii. Input the convolutional features into the long short-term memory network (LSTM) modeling module to obtain the hidden state of the preset training joint at the current moment.

[0086] In the embodiment of the present application, substitute the convolutional features into the following formula to obtain the hidden state of the preset training joint at the current moment:

[0087]

[0088] where h t is the hidden state of the preset training joint at the current moment t, LSTM is the long short-term memory network, y t is the convolutional feature at the current moment t, is the optimized hidden state of the preset training joint at the moment t - 1 (used to capture historical time series information), c t-1 is the cell state of the preset training joint at the moment t - 1 (used for long-term dependence modeling).

[0089] Meanwhile, optimize the gradient propagation of the LSTM through residual connection. The residual learning formula is as follows:

[0090]

[0091] where is the optimized hidden state of the current training joint at the moment t, and α is the residual weight coefficient.

[0092] iii. Input the hidden state into the attention module to obtain the second joint angle of the preset training joint at the current moment.

[0093] In the embodiment of the present application, substitute the hidden state into the following formula to obtain the second joint angle of the preset training joint at the current moment:

[0094]

[0095] θ t = ∑ t′ α t′ h t′ ;

[0096] where α t is the attention weight at the current moment t, v is the attention vector (a parameter of the joint angle estimation model), W′ is the weight matrix (a parameter of the joint angle estimation model), b is the bias vector (enhancing the nonlinear fitting ability of the model), h t′ is the hidden state at the moment t′, θ t is the second joint angle of the preset training joint at the current moment t, t′ is any moment between the moment 0 and the moment t, α t′is the attention weight at time t′ (indicating the contribution degree of time t′ to the current time t).

[0097] S203. Calculate the target joint angle of the preset training joint at the current time according to the first joint angle and the second joint angle.

[0098] In the embodiment of the present application, substitute the first joint angle and the second joint angle into the following formula to obtain the target joint angle of the preset training joint at the current time;

[0099] g = σ(W g ·[θ init (t), θ t + b g )

[0100] θ final (t) = g·θ inut (t) + (1 - g)·θ t ;

[0101] where g is the gating mechanism, σ is the activation function, W g is the weight matrix, θ init (t) is the first joint angle at the current time t, θ t is the second joint angle at the current time t, b g is the bias, and θ final (t) is the target joint angle at the current time t.

[0102] Step 2. Generate the motion trajectory of the virtual digital human from the previous time to the current time according to the motion data within the current time period and the joint angle at the current time.

[0103] In the embodiment of the present application, use a 3D modeling tool to design the virtual digital human of the patient. The virtual digital human includes the complete bone structure of the torso, limbs and head. Based on the position information of the virtual digital human from the previous time, the motion data and joint angle at the current time, calculate the position information of the virtual digital human at the current time; use the curve interpolation algorithm to generate the motion trajectory of the virtual digital human (i.e., the motion trajectory of the patient from the previous time to the current time) according to the position information of the virtual digital human from the previous time, the motion data, joint angle and position information at the current time.

[0104] Here, the client of the patient can be devices such as a tablet or a mobile phone. Under the support of the rendering engine, the virtual digital human is updated in real time at a frequency of 60 frames per second to ensure smooth action synchronization. The algorithm for calculating the position information of the virtual digital human at the current time can be the Kalman filtering algorithm, etc.

[0105] S103. Compare the virtual digital human with the demonstrator in the simultaneously played standard training video according to the preset rehabilitation training action parameters corresponding to the preset training joints.

[0106] In the embodiment of the present application, the comparison result may include the deviation value between the parameter value corresponding to the preset rehabilitation training action parameter of the virtual digital human corresponding to the patient and the parameter value corresponding to the preset rehabilitation training action parameter of the demonstrator. The comparison result may also include training results such as non-standard or standard preset rehabilitation training actions, and incorrect preset rehabilitation training actions.

[0107] Here, different joints correspond to different preset rehabilitation training action parameters; the preset rehabilitation training action parameters corresponding to the shoulder joint may include the joint angle of the shoulder joint, the movement trajectory of the shoulder joint, the abduction height of the shoulder, the arm lift angle, the arm rotation amplitude, etc. The preset rehabilitation training action parameters corresponding to the elbow joint may include the joint angle of the elbow joint. The preset rehabilitation training action parameters corresponding to the hip joint may include the range of motion of the hip joint, the rotation angle of the hip joint, and the movement trajectory of the hip joint. The preset rehabilitation training action parameters corresponding to the knee joint may include the joint angle of the knee joint. The preset rehabilitation training action parameters corresponding to the ankle joint may include the rotation angle and rotation range of the ankle joint.

[0108] S104. Provide real-time prompts for the patient's joint rehabilitation training according to the comparison result.

[0109] In the embodiment of the present application, the joint rehabilitation training prompts include color highlighting prompts for the action deviation area on the virtual digital human corresponding to the patient, voice prompts, vibration prompts, text prompts, etc.

[0110] Providing real-time prompts for the patient's joint rehabilitation training according to the comparison result includes: highlighting the deviation area between the virtual digital human corresponding to the patient and the demonstrator by color highlighting to provide intuitive visual feedback.

[0111] Providing real-time prompts for the patient's joint rehabilitation training according to the comparison result also includes: performing voice prompts according to the prompt text corresponding to the preset range where the deviation value is located; different preset prompt texts correspond to different preset ranges where the parameter value is located.

[0112] For example, if the preset range of the deviation value indicates problems such as insufficient abduction height, deviation in the raising angle, or insufficient rotation amplitude of the patient's shoulder joint, real-time reminders will be provided to the patient to correct the posture to ensure the training effect. If the preset range of the deviation value indicates insufficient flexion and extension, excessive force, or overly hasty movements of the patient's elbow joint, adjustment suggestions will be given to remind the patient to relax and complete the movement at an appropriate speed. This training helps to restore the flexibility of the elbow joint and cultivate the control ability of joint movement. If the preset range of the deviation value indicates asymmetry or deviation from the standard of the hip joint, immediate feedback guidance will be provided to help the patient adjust the hip posture to gradually restore the function of the hip joint and reduce the negative impact of incorrect postures. If the preset range of the deviation value indicates insufficient knee flexion, insufficient extension angle, or excessive flexion, the patient will be reminded to make adjustments to ensure the accuracy of the movement and prevent excessive joint pressure caused by overtraining. If the preset range of the deviation value indicates insufficient range of motion or improper force of the ankle joint, the system will immediately provide feedback to remind the patient to relax the ankle or adjust the angle to avoid training injuries caused by incorrect postures.

[0113] In addition, the method further includes: constructing different rehabilitation training scenarios through virtual reality technology, such as at home, in a sports field, etc., and the patient can complete specific rehabilitation tasks in an immersive environment. For example, in the "at home" scenario, the patient can simulate picking up items or arranging the positions of objects to complete training tasks related to daily life. The logic script is bound to the rehabilitation training scenario in real time to drive the change of the target object in the rehabilitation training scenario according to the patient's actual actions, so as to achieve the deep integration of rehabilitation training and daily life situations.

[0114] To further improve the personalization and adaptability of the training, the method further includes: adjusting the training parameters of the rehabilitation training actions of the demonstrator according to the training parameters of the patient for the rehabilitation training actions in the preset training video; the training parameters of the rehabilitation training actions include the completion speed of the rehabilitation training actions, the amplitude of the rehabilitation training actions, etc. For example, as the patient's training ability improves, the system can gradually increase the amplitude of the rehabilitation training actions, extend the holding time of the rehabilitation training actions, or increase the requirements for the continuity of the actions. At the same time, the patient's training data is recorded and uploaded to the server to provide detailed progress reports and data analysis support for doctors, enabling the training plan to be flexibly optimized according to the actual needs of the patient.

[0115] S105. After the training is completed, generate a rehabilitation report for the patient according to the actual training index parameters during the limb rehabilitation training process and the large language model obtained by using the fine-tuning technology; the actual training index parameters are determined based on all the comparison results during the training process.

[0116] In the embodiments of the present application, according to the actual training index parameters, the preset training index parameters, and the preset joint rehabilitation report generation rules, prompt words are generated; the prompt words are input into a large language model to obtain the patient's rehabilitation report.

[0117] In the embodiments of the present application, the large language model is obtained by fine-tuning training with prompt word samples and corresponding rehabilitation reports; the prompt word samples are generated based on the actual training index parameter samples, the preset training index parameter samples, and the preset joint rehabilitation report generation rule samples during the patient's rehabilitation training process.

[0118] Here, the actual training index parameters include the actual training completion duration, actual accuracy, actual deviation range, and actual error frequency of each rehabilitation training action; the preset training index parameters include the preset training completion duration, preset accuracy, preset deviation range, and preset error frequency of each rehabilitation training action.

[0119] Furthermore, the patient's rehabilitation report includes the rehabilitation trend (the large language model analyzes the patient's rehabilitation progress by comparing the patient's actual training index parameters with the initial training index parameters, and can present the curve trends of key indicators (such as action completion rate, accuracy rate) changing over time using charts), the training problems existing in the patient's rehabilitation training process, the adjustment suggestions for the preset training index parameters, the evaluation results of the effectiveness of the preset training plan (combining the actual training index parameters and the preset training index parameters to evaluate whether the preset training plan achieves the expected effect, and recommending feasible optimization directions (such as adjusting the task difficulty, modifying the training frequency, etc.) when necessary), the training risk prediction results, the training risk prevention measures, etc.

[0120] In addition, to meet the needs of different users, the report content supports dual output of specialization and personalization. The rehabilitation report for the doctor side mainly uses medical professional terms and standardized quantitative indicators, which is convenient for doctors to quickly understand and use for decision support; the rehabilitation report for the patient side simplifies the expression form, accompanied by visual charts and easy-to-understand text descriptions, enabling patients to intuitively understand the rehabilitation progress and future directions. For example, complex quantitative scores can be converted into simple labels such as "excellent, good, medium, poor", enhancing the patient's sense of participation and confidence.

[0121] The embodiment of the present application provides a limb rehabilitation training method and system based on digital twin and AI technologies. The method includes: in response to a patient's limb rehabilitation training request, playing a standard training video of a preset training joint corresponding to the patient's current training limb on the patient's client, so that the patient can train according to the standard training video; and acquiring the motion data of the preset training joint in the current time period in real time; using digital twin technology, generating the motion trajectory of the virtual digital person shown on the patient's client from the previous moment to the current moment according to the motion data in the current time period, and controlling the virtual digital person according to the motion trajectory; comparing the virtual digital person with the demonstrator in the simultaneously played standard training video according to the preset rehabilitation training action parameters corresponding to the preset training joint; giving real-time prompts for the patient's joint rehabilitation training according to the comparison result; after the training is completed, generating a rehabilitation report for the patient according to the actual training index parameters during the limb rehabilitation training process of the patient and the large language model obtained by using the fine-tuning technology; the actual training index parameters are determined based on all the comparison results during the training process. Based on digital twin and AI technologies, the motion data of the patient is processed in real time, and the virtual digital person is controlled in real time based on this motion data to give real-time prompts for joint rehabilitation training, so as to be able to monitor the patient's limb rehabilitation training process in real time, and use AI technology to generate a personalized rehabilitation report in time, thereby shortening the patient's rehabilitation period.

[0122] Here, based on the same inventive concept, the embodiment of the present application also provides a limb rehabilitation training system based on digital twin and AI technologies corresponding to the limb rehabilitation training method based on digital twin and AI technologies. Since the principle of solving problems by the system in the embodiment of the present application is similar to that of the above-mentioned limb rehabilitation training method based on digital twin and AI technologies in the embodiment of the present application, the implementation of the system can refer to the implementation of the method, and the repeated parts will not be described again.

[0123] Referring to Figure 3 As shown in

[0124] A response module 301, configured to, in response to a patient's limb rehabilitation training request, play a standard training video of a preset training joint corresponding to the patient's current training limb on the patient's client, so that the patient can train according to the standard training video; and acquire the motion data of the preset training joint in the current time period in real time;

[0125] A generation control module 302, configured to use digital twin technology to generate the motion trajectory of the virtual digital person shown on the patient's client from the previous moment to the current moment according to the motion data in the current time period, and control the virtual digital person according to the motion trajectory;

[0126] A comparison module 303 is configured to compare the virtual digital human with the demonstrator in the simultaneously played standard training video according to the preset rehabilitation training action parameters corresponding to the preset training joints;

[0127] A prompt module 304 is configured to perform real-time prompting for the patient's joint rehabilitation training according to the comparison result;

[0128] A generation module 305 is configured to generate a rehabilitation report for the patient based on the actual training index parameters of the patient during the limb rehabilitation training process and the large language model obtained by using the fine-tuning technology after the training is completed; the actual training index parameters are determined based on all the comparison results during the training process.

[0129] The embodiment of the present application provides a limb rehabilitation training system based on digital twin and AI technologies. This system is based on digital twin and AI technologies, processes the patient's motion data in real time, controls the virtual digital human to perform real-time joint rehabilitation training prompts based on this motion data, so as to be able to monitor the patient's limb rehabilitation training process in real time, and uses AI technologies to generate personalized rehabilitation reports in a timely manner, thereby shortening the patient's rehabilitation cycle.

[0130] As Figure 4 shown, an electronic device 400 provided by the embodiment of the present application includes: a processor 401, a memory 402, and a bus. The memory 402 stores machine-readable instructions executable by the processor 401. When the electronic device runs, the processor 401 communicates with the memory 402 through the bus, and the processor 401 executes the machine-readable instructions to perform the steps of the above-mentioned limb rehabilitation training method based on digital twin and AI technologies.

[0131] Specifically, the above-mentioned memory 402 and processor 401 can be general-purpose memory and processor, which are not specifically limited here. When the processor 401 runs the computer program stored in the memory 402, it can execute the above-mentioned limb rehabilitation training method based on digital twin and AI technologies.

[0132] Corresponding to the above-mentioned limb rehabilitation training method based on digital twin and AI technologies, the embodiment of the present application also provides a computer-readable storage medium. A computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, it executes the steps of the above-mentioned limb rehabilitation training method based on digital twin and AI technologies.

[0133] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems and devices described above can refer to the corresponding processes in the method embodiments, and will not be elaborated herein. In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division, and there can be other division methods in actual implementation. For another example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the couplings, direct couplings, or communication connections shown or discussed among each other can be through some communication interfaces. The indirect couplings or communication connections of the devices or modules can be in electrical, mechanical, or other forms.

[0134] The modules described as separate components may or may not be physically separated. The components shown as modules may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0135] In addition, in each embodiment of the present application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0136] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the information processing method described in each embodiment of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical discs that can store program codes.

[0137] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, and all should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A limb rehabilitation training method based on digital twin and AI technology, characterized in that: The method comprises: In response to a patient's request for limb rehabilitation training, a standard training video of a preset training joint corresponding to the patient's current training limb is played on the patient's client, so that the patient can train according to the standard training video; and the motion data of the preset training joint in the current time period is obtained in real time; Using digital twin technology, based on the motion data in the current time period, a motion trajectory of a virtual digital human from the previous moment to the current moment is generated and displayed on the client of the patient, and the virtual digital human is controlled according to the motion trajectory; According to the preset rehabilitation training action parameters corresponding to the preset training joints, the virtual digital human is compared with the demonstrator in the simultaneously played standard training video; Providing real-time joint rehabilitation training to the patient according to the comparison results; After the training is completed, a rehabilitation report for the patient is generated based on the actual training index parameters of the patient during the limb rehabilitation training and the large language model obtained using fine-tuning technology; the actual training index parameters are determined based on all the comparison results during the training process.

2. The limb rehabilitation training method based on digital twin and AI technology according to claim 1, characterized in that: The step of generating a motion trajectory of a virtual digital person from a previous moment to a current moment displayed on a client of the patient according to the motion data in the current time period includes: Determining the joint angle of the preset training joint at the current moment according to the motion data in the current time period; The motion trajectory of the virtual digital human from the previous moment to the current moment is generated according to the motion data in the current time period and the joint angle at the current moment.

3. The limb rehabilitation training method based on digital twin and AI technology according to claim 1, characterized in that: The step of determining the joint angle of the preset training joint at the current moment according to the motion data in the current time period includes: Calculating the first joint angle of the preset training joint at the current moment according to the motion data at the current moment; Inputting the motion data in the current time period into a joint angle estimation model to obtain a second joint angle of the preset training joint at the current moment; According to the first joint angle and the second joint angle, a target joint angle of the preset training joint at the current moment is calculated.

4. The limb rehabilitation training method based on digital twin and AI technology according to claim 3, characterized in that: The motion data includes motion data of each preset body part in the preset body part pair corresponding to the preset training joint; The step of calculating the first joint angle of the preset training joint at the current moment according to the motion data at the current moment comprises: For each of the preset body parts, calculating a posture vector of the preset body part at the current moment according to the motion data of the preset body part at the current moment; According to the posture vectors of all the preset body parts at the current moment, the first joint angle of the preset training joint at the current moment is calculated.

5. The limb rehabilitation training method based on digital twin and AI technology according to claim 4, characterized in that: The step of calculating the first joint angle of the preset training joint at the current moment according to the posture vectors of all the preset body parts at the current moment comprises: Substitute the posture vectors of all the preset body parts at the current moment into the following formula to obtain the first joint angle of the preset training joint at the current moment: Among them, θ init (t) is the first joint angle of the preset training joint at the current moment, is the posture vector of the first preset body part in the preset body part pair at the current moment, The posture vector of the second preset body part at the current moment for the preset body part alignment, The posture vector of the first preset body part at the current moment for the preset body part alignment The model, The pose vector of the second preset body part at the current moment for the preset body part to be centered Model.

6. The limb rehabilitation training method based on digital twin and AI technology according to claim 3, characterized in that: The step of calculating the target joint angle of the preset training joint at the current moment according to the first joint angle and the second joint angle includes: Substituting the first joint angle and the second joint angle into the following formula, the target joint angle of the preset training joint at the current moment is obtained; g=σ(W g ·[θ init (t),θ t ]+b g ); i final (t)=g·θ init (t)+(1-g)·θ t ; Among them, g is the gating mechanism, σ is the activation function, and W g is the weight matrix, θ init (t) is the first joint angle at the current time t, θ t is the second joint angle at the current time t, b g is the bias, θ final (t) is the target joint angle at the current time t.

7. The limb rehabilitation training method based on digital twin and AI technology according to claim 1, characterized in that: The method further comprises: Generate prompt words according to the actual training index parameters, the preset training index parameters and the preset joint rehabilitation report generation rules; The prompt words are input into a large language model to obtain a rehabilitation report of the patient.

8. A limb rehabilitation training system based on digital twin and AI technology, characterized in that: The system comprises: A response module is used to respond to a patient's request for limb rehabilitation training, play a standard training video of a preset training joint corresponding to the patient's current training limb on the patient's client, so that the patient can train according to the standard training video; and obtain the motion data of the preset training joint in the current time period in real time; A generation control module is used to generate a motion trajectory of a virtual digital human from a previous moment to a current moment displayed on the client of the patient based on the motion data in the current time period by using the digital twin technology, and control the virtual digital human according to the motion trajectory; A comparison module, for comparing the virtual digital human with the demonstrator in the standard training video played simultaneously according to the preset rehabilitation training action parameters corresponding to the preset training joints; A prompt module, used for providing real-time prompts for joint rehabilitation training to the patient according to the comparison results; A generation module is used to generate a rehabilitation report for the patient after the training is completed, based on the actual training index parameters of the patient during the limb rehabilitation training and the large language model obtained using the fine-tuning technology; the actual training index parameters are determined based on all the comparison results during the training process.

9. An electronic device, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to perform the steps of the limb rehabilitation training method based on digital twins and AI technology as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the limb rehabilitation training method based on digital twin and AI technology as described in any one of claims 1 to 7 are executed.

Images