Virtual Reality-Based Rehabilitation Training Evaluation Method, Device, Equipment and Medium

Abstract

The present disclosure relates to the technical field of rehabilitation training, and in particular, to a rehabilitation training evaluation method, device, equipment, and medium based on virtual reality. The method includes: displaying a virtual training scene corresponding to a training user, where a virtual prop and a first virtual human model are presented in the virtual training scene; acquiring the motion data of the training user; controlling the first virtual human model to perform synchronous motion based on the motion data, and controlling the first virtual human model to strike the virtual prop; during the rehabilitation training process, analyzing and statistically processing the motion data of the training user to obtain a rehabilitation evaluation result. It realizes helping patients to perform efficient rehabilitation training that conforms to the motion laws of actual props in a virtual reality environment, so as to meet personalized needs and improve the fun of training.

Description

Technical Field

[0001] The present disclosure relates to the technical field of rehabilitation training, and in particular, to a rehabilitation training evaluation method, device, equipment, and medium based on virtual reality. Background Art

[0002] With the growth of the elderly population, stroke has gradually attracted attention due to its characteristics of high incidence, high disability rate, and high mortality rate. Scientific rehabilitation training plays an important role in the limb function rehabilitation of stroke hemiplegic patients.

[0003] Virtual reality (VR) technology is a technology that uses a computer to generate a virtual environment that simulates real things (such as walking, picking up objects, etc.), and through specific interaction tools, such as stereoscopic glasses, sensing gloves, etc., enables the experiencer to "immerse" in the environment and realize the natural interaction between the experiencer and the virtual environment directly. The virtual reality system can make the user feel the degree of authenticity of being the protagonist in the simulated environment, just like the feeling in the real world.

[0004] In recent years, virtual reality technology has begun to be applied in limb function rehabilitation training. Let patients complete controllable functional movements and operations in a virtual environment to achieve the purpose of functional reconstruction. By using the virtual environment to encourage patients to actively participate in rehabilitation training, understand the laws of rehabilitation, and formulate training goals, not only can the training purpose be achieved, but also it helps to improve the effect of rehabilitation training.

[0005] However, in related technologies, rehabilitation trainers follow some prescribed actions in virtual games, and the training methods are single and cannot meet personalized needs. Summary of the Invention

[0006] To solve the above-mentioned all or at least one technical problem, the embodiments of the present disclosure provide a rehabilitation training evaluation method, device, equipment, and medium based on virtual reality, which helps patients to perform efficient rehabilitation training that conforms to the movement laws of actual props in a virtual reality environment to meet personalized needs and improve the fun of training.

[0007] In a first aspect, the embodiments of the present disclosure provide a rehabilitation training evaluation method based on virtual reality, including: displaying a virtual training scene corresponding to a training user, where a virtual prop and a first virtual human model are presented in the virtual training scene; acquiring the action data of the training user; controlling the first virtual human model to perform synchronous movement based on the action data, and controlling the first virtual human model to strike the virtual prop; during the rehabilitation training process, analyzing and statistically processing the action data of the training user to obtain a rehabilitation evaluation result.

[0008] In a second aspect, an embodiment of the present disclosure provides a rehabilitation training evaluation device based on virtual reality, including: a virtual training scenario display module for displaying a virtual training scenario corresponding to a training user, in which virtual props and a first virtual human model are presented; an action data acquisition module for acquiring the action data of the training user; a first model control module for controlling the first virtual human model to perform synchronous movement based on the action data and controlling the first virtual human model to strike the virtual props; and a rehabilitation evaluation module for analyzing and statistically processing the action data of the training user during the rehabilitation training process to obtain a rehabilitation evaluation result.

[0009] In a third aspect, an embodiment of the present disclosure provides an electronic device, which includes:

[0010] One or more processors;

[0011] A storage device for storing one or more programs;

[0012] When the one or more programs are executed by the one or more processors, the one or more processors implement the virtual reality-based rehabilitation training evaluation method provided in any one of the above first aspects.

[0013] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the virtual reality-based rehabilitation training evaluation method provided in any one of the above first aspects.

[0014] In a fifth aspect, an embodiment of the present disclosure provides a program product, which includes a computer program, and when the computer program is executed by a processor, it implements the virtual reality-based rehabilitation training evaluation method provided in any one of the above first aspects.

[0015] Embodiments of the present disclosure provide a rehabilitation training evaluation method, apparatus, device, and medium based on virtual reality. The method includes: presenting a virtual training scene corresponding to a training user, where a virtual prop and a first virtual human model are presented in the virtual training scene; obtaining the motion data of the training user; controlling the first virtual human model to perform synchronous motion based on the motion data, and controlling the first virtual human model to strike the virtual prop; during the rehabilitation training process, analyzing and statistically processing the motion data of the training user to obtain a rehabilitation evaluation result. Since during the process of the training user performing rehabilitation training in the virtual training scene, the motion data of the training user can be collected, and based on this motion data, the first virtual human model in the virtual training scene is controlled to perform synchronous actions, and when performing synchronous actions, the virtual prop is struck, it helps the patient to perform efficient rehabilitation training that conforms to the actual motion law of the prop in the virtual reality environment to meet personalized needs and improve the fun of training. And the key training data during the rehabilitation training process is statistically processed, and the rehabilitation situation is evaluated based on the key training data, so as to provide data support for formulating a personalized training method for this user in the future. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In combination with the accompanying drawings and with reference to the following specific embodiments, the above and other features, advantages, and aspects of the embodiments of the present disclosure will become more obvious. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic, and the original components and elements are not necessarily drawn to scale.

[0017] Figure 1 It is a flowchart of a rehabilitation training evaluation method based on virtual reality in an embodiment of the present disclosure;

[0018] Figure 2 It is a flowchart of a motion data acquisition method in an embodiment of the present disclosure;

[0019] Figure 3 It is a data diagram for controlling virtual tennis movement in an embodiment of the present disclosure;

[0020] Figure 4 It is a schematic diagram of the line change of a tennis match in an embodiment of the present disclosure;

[0021] Figure 5 It is a schematic structural diagram of a rehabilitation training evaluation apparatus based on virtual reality in an embodiment of the present disclosure;

[0022] Figure 6 It is a schematic structural diagram of an electronic device in an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0024] It should be understood that the various steps described in the method embodiments of the present disclosure can be executed in different orders and / or executed in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this regard.

[0025] The term "including" and its variations used herein are open-ended, that is, "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0026] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order of functions executed by these devices, modules or units or their interdependent relationships.

[0027] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly specified in the context, it should be understood as "one or more".

[0028] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0029] In recent years, virtual reality technology has begun to be applied in limb function rehabilitation training. Let patients complete controllable functional movements and operations in a virtual environment to achieve the purpose of functional reconstruction. By using a virtual environment to encourage patients, enabling them to actively participate in rehabilitation training, understand the laws of rehabilitation, and set training goals, not only can the training purpose be achieved, but also it helps to improve the effect of rehabilitation training.

[0030] However, in the related art, rehabilitation trainers follow virtual games to perform some prescribed actions. For example: prompting the training user to perform an arm-lifting action, prompting the training user to perform a kicking action, and so on. This training method is single and cannot meet personalized needs.

[0031] To solve all or at least one of the above technical problems, embodiments of the present disclosure provide a rehabilitation training evaluation method, apparatus, device, and medium based on virtual reality. The method includes: presenting a virtual training scene corresponding to a training user, where a virtual prop and a first virtual human model are presented in the virtual training scene; obtaining the motion data of the training user; controlling the first virtual human model to perform synchronous motion based on the motion data, and controlling the first virtual human model to strike the virtual prop; and analyzing and statistically processing the motion data of the training user during the rehabilitation training process to obtain a rehabilitation training result.

[0032] Since during the process of the training user performing rehabilitation training in the virtual training scene, the motion data of the training user can be collected, and based on the motion data, the first virtual human model in the virtual training scene is controlled to perform synchronous actions, and when performing synchronous actions, the virtual prop is struck, which helps the patient to perform efficient rehabilitation training that conforms to the actual motion law of the prop in the virtual reality environment to meet personalized needs and improve the fun of training. And the key training data during the rehabilitation training process is statistically processed, and the rehabilitation situation is evaluated based on the key training data, so as to provide data support for formulating a personalized training method for this user in the future.

[0033] The following will introduce in detail the rehabilitation training evaluation method based on virtual reality provided by the present disclosure in combination with embodiments and drawings.

[0034] Figure 1 It is a flowchart of a rehabilitation training evaluation method based on virtual reality in embodiments of the present disclosure. This embodiment is applicable to the situation of using virtual reality technology for limb function rehabilitation training. The method can be executed by a rehabilitation training evaluation device based on virtual reality. The rehabilitation training evaluation device based on virtual reality can be implemented in a software and / or hardware manner, and the rehabilitation training evaluation device based on virtual reality can be configured in a virtual reality device.

[0035] As Figure 1 shown, the rehabilitation training evaluation method based on virtual reality provided by embodiments of the present disclosure mainly includes steps S101 - S104.

[0036] S101. Present a virtual training scene corresponding to a training user, where a virtual prop and a first virtual human model are presented in the virtual training scene.

[0037] The training user can be understood as a user who needs to perform limb function rehabilitation training. The limb function rehabilitation training can be upper limb function rehabilitation training or lower limb function rehabilitation training. The virtual training scene can be understood as a simulation environment provided by a virtual reality system for the training user, which can make the user feel like the protagonist.

[0038] Among them, virtual props and a first virtual human model are presented in the virtual training scenario. The first virtual human model is the virtual human model corresponding to the training user in the virtual training scenario. In other words, the first virtual human model performs the same actions as the training user in the virtual training scenario. The virtual props refer to the virtual props that the training user needs to use during the rehabilitation training process. For example, the virtual props can be virtual tennis balls, virtual table tennis balls, virtual basketballs, virtual footballs, etc. In this embodiment, the virtual prop is a virtual tennis ball as an example for illustration.

[0039] Based on the basic information of the training user, determine the training item corresponding to the training user, obtain the virtual scene data corresponding to the training item, and render the virtual scene data to obtain a virtual training scenario.

[0040] Furthermore, select the first virtual human model from multiple virtual human models based on the identifier of the training user. Among them, the identifier of the training user can be the identifier of the sensor worn by the training user.

[0041] The basic information of the training user may include the identity information and basic physical signs information of the training user. Among them, the identity information may include information such as name, training code, and hospitalization code that characterize the identity of the training user. The basic physical signs information may include information such as height, weight, blood pressure, and pulse that characterize the physical state. The rehabilitation data of the patient is stored in the patient database, and the training data includes information related to training such as the training items the patient needs, training content, training times, and the corresponding training results each time.

[0042] In an exemplary implementation, obtain the basic information of the training user in response to the input operation of the doctor. Among them, the input operation can be that the doctor manually enters the basic information of the training user, or the basic information of the training user is read by reading the medical card of the training user.

[0043] After obtaining the basic information of the training user, query the rehabilitation data corresponding to the training user in the patient database based on the identity information in the basic information. After obtaining the rehabilitation data, generate and display at least one training information of the training user based on the rehabilitation data and the basic information.

[0044] The training information includes the training item and at least one training difficulty included in the training item, and the training item and at least one training difficulty included in the training item are displayed on the interface of the training client. The doctor can select the corresponding training item and training level according to the displayed training item and the training difficulty included in the training item, and determine the corresponding virtual scene data based on the training item and the training difficulty, and render the virtual scene data.

[0045] S102. Obtain the motion data of the training user.

[0046] The motion data of the training user mainly refers to the relevant data representing the limb motions performed by the training user during the rehabilitation training. Further, the motion data mainly includes the position data and rotation data of the training user. The position data refers to the positions where the main joints of the user are located, and the rotation data refers to the angles of rotation of the main joints or bones of the user.

[0047] A data tracker is worn on the body of the training user, and the data tracker may include an inertial sensor and / or an optical sensor. The data acquisition device includes a corresponding optical camera and an inertial receiver. The inertial receiver is used to acquire the inertial data emitted by the inertial sensor; the optical camera is used to acquire the optical data reflected by the optical reflector ball.

[0048] The inertial data and / or optical data are obtained from the data acquisition device by means of direct address connection, and the inertial data and / or optical data are continuously obtained at 30 frames per second. By performing fusion calculation on the inertial data and the optical data, the motion data corresponding to a position can be obtained.

[0049] Among them, the camera uses a programmable computer controller (PCC) optical tracking camera. The PCC optical tracking camera designs an external interface, and this interface provides functions of 10 / 100M network input / output and power input.

[0050] The PCC optical tracking camera is based on a high-performance image processing chip and is equipped with ten ultra-high-power infrared LEDs. It can provide image processing capabilities with a maximum resolution of 1280*1024 and 200 frames. It can capture the two-dimensional position information of the optical inertial tracker on the target object and output it to the data mixing unit. The data mixing unit calculates the three-dimensional position information and motion posture of the target in space, realizes large-space position and posture capture, and is applied in fields such as large-space multi-person VR, motion capture, and medical rehabilitation surgery.

[0051] The data acquisition device is used for tracking the position and posture of the training user. The data acquisition device is specifically composed of several PCC optical lenses, a switch, a data processing host, a calibration tool, an inertial receiver, and a multi-source data acquisition and tracking service system.

[0052] The optical data and inertial data of the training user in the acquisition activity area are collected. The PCC optical lenses and the inertial receiver are connected to the switch of the network device through network cables. The switch assigns the corresponding LAN IP addresses to the above devices, obtains its picture through the IP address of the PCC optical lens, and similarly receives the inertial data through the IP of the inertial receiver.

[0053] The data tracker adopts a detachable split design concept of an optical rigid body and an inertial sensor. The use of the K-type rigid body directly solves the problem of the limited number of rigid body tracking in traditional optical systems. At the same time, combined with the hybrid algorithm of inertial data, it makes up for the disadvantages of pure optical rigid bodies being easily blocked and data loss, and also solves the problem of cumulative errors of inertial sensors, which can greatly improve the tracking stability and continuity of the tracker, so as to achieve the effect of dense and stable tracking of multiple people and multiple props.

[0054] Using the human motion data captured by the optical tracker or inertial sensor, through high-frequency data sampling, the three-dimensional space motion information of the human body is converted into a digital data stream. These data go through preprocessing steps such as filtering and calibration to ensure data accuracy and stability.

[0055] In a possible implementation, the device identifier is listened to and responded to in real time. When the listened device identifier is a pre-configured device identifier, the name of the corresponding training user and the first virtual human model are created in the virtual training scenario. Among them, the device identifier can be the identifier of the inertial sensor, the identifier of the optical tracker or the identifier of the K-type rigid body.

[0056] The received motion data includes optical data and inertial data. Among them, the motion data includes head data, back data, left hand data, right hand data, left foot data, right foot data, and the motion data of 5 fingers of the left hand and 5 fingers of the right hand. Each position is distinguished by a different name identifier. Among them, the optical data is distinguished by the type "rigidType", and the inertial data is distinguished by the type "sensor". Each motion data contains 4 sets of coordinates. Using these 4 sets of coordinates and the coordinate transformation relationship between the model coordinate system and the optical coordinate system, the real-time motion data in the model coordinate system is obtained. In other words, the optical data includes 4 sets of coordinates, and the inertial data includes 4 sets of coordinates.

[0057] Taking the optical data as an example below, the determination method of the coordinate transformation relationship between the model coordinate system and the optical coordinate system is described.

[0058] First, solve the centroid (average position).

[0059] For the optical data of each position, the average position corresponding to the set is calculated by traversing the list and accumulating the position information of all model marker points, and then divided by the number of model marker points to obtain the actual average position of the model marker points. The average position corresponding to the set is calculated by traversing the list and accumulating the position information of all optical marker points, and then divided by the number of optical marker points to obtain the actual average position of the optical marker points.

[0060] Second, sort the marker points.

[0061] Create a new first list pas and a second list pbs. Among them, the first list pas is a list storing model marker point information, where the model marker point information refers to the squared value of the centroid distance. Further, the first list pas is a list of model marker points sorted in descending order of the squared value of the centroid distance. Among them, the second list pbs is a list storing optical marker point information, where the optical marker point information refers to the squared value of the centroid distance. Further, the second list pbs is a list of optical marker points sorted in descending order of the squared value of the centroid distance.

[0062] This can ensure that three representative points are selected for constructing the coordinate axes.

[0063] Third, calculate the model coordinate axes.

[0064] Use the sorted first list pas to calculate the Z-axis vector (modeldirZ) and Y-axis vector (modeldirY) of the model. Optionally, a right-handed coordinate system is adopted, the Z-axis is the unit vector of the line connecting two points, and the Y-axis is the unit vector perpendicular to the screen found through cross multiplication. Then use the LookRotation function to create a quaternion modelQ representing the rotation of the model coordinate system.

[0065] Fourth, calculate the optical coordinate axes.

[0066] Use the sorted second list pbs to calculate the Z-axis vector (alicedirZ) and Y-axis vector (alicedirY) of the optics. Optionally, a right-handed coordinate system is adopted, the Z-axis is the unit vector of the line connecting two points, and the Y-axis is the unit vector perpendicular to the screen found through cross multiplication. Then use the LookRotation function to create a quaternion aliceQ representing the rotation of the optical coordinate system.

[0067] Fifth, convert from the model space to the optical space.

[0068] First, create a first matrix MA and a second matrix MB. Among them, the first matrix MA represents the conversion from the model coordinate system to the world coordinate system, and the second matrix MB represents the conversion from the optical coordinate system to the world coordinate. Then calculate the conversion matrix M_A_TO_B from the model coordinate system to the optical coordinate system.

[0069] Sixth, extract the position and rotation.

[0070] Extract the translation part from the M_A_TO_B matrix as the new position pos (i.e., the fourth column), and obtain the rotation part by making a LookRotation function call. Here, the second column (the original Z-axis direction of the model) is used as the target direction of rotation, and the first column (the original Y-axis direction of the model) is used as the up vector to determine the final rotation quaternion rot. The purpose of the entire function is, given two three-dimensional point sets with the same structure (each point set contains at least three non-collinear points), to find an optimal fitting transformation from the model coordinate system to the optical coordinate system by comparing and transforming these two point sets, so as to be able to describe the object state in one coordinate system in terms of the other coordinate system.

[0071] Taking the point (0, 0, 0) in the virtual training scene as the origin of the space coordinate system, the received rigid body data uses the coordinate origin as the reference object to establish its relative spatial relationship and complete registration.

[0072] It should be noted that in the above embodiments, taking the optical data as an example, the fitting transformation relationship between the model coordinate system and the optical coordinate system is introduced. Further, the fitting transformation relationship between the model coordinate and the inertial coordinate system can be established in the above manner.

[0073] Using the 4 sets of coordinates included in the optical data and the fitting transformation relationship between the model coordinate system and the optical coordinate system, calculate the optical data in the model coordinate system. Using the 4 sets of coordinates included in the inertial data and the fitting transformation relationship between the model coordinate system and the inertial coordinate system, calculate the inertial data in the model coordinate system.

[0074] Further, fuse the optical data and the inertial data to obtain the action data corresponding to each position in the model coordinate system. Among them, the action data includes position data and rotation data. To ensure the stability of the data, it is necessary to perform smoothing filtering on the continuous position (pos) and rotation (rot) data.

[0075] Specifically, collect the action data of the training user, including: for the action data of any joint point, determine whether the difference between the current action data and the average action data is greater than the set threshold; when the difference is less than or equal to the set threshold, enter the real-time sensitive state; in the real-time sensitive state, initialize the timer and increase the time value in units of each frame; when the time value of the timer is less than or equal to the set time value, assign the currently obtained action data to the average action data; when the time value of the timer is greater than the set time value, reset the timer and exit the real-time sensitive state; when the difference is greater than the set threshold, enter the smooth tracking state; in the smooth tracking state, initialize the index and fill the current action data into the historical data array; calculate the average action data from all the action data in the historical data array.

[0076] The process of smoothing the motion data can be divided into two main stages: real-time sensitive state and smooth tracking state.

[0077] Compare whether the squared difference between the average position data and the current position data exceeds a first preset threshold, and whether the angular difference between the average rotation data and the current rotation exceeds a second preset threshold. If both exceed, restart the real-time sensitive state and update the average position data and average rotation data to the current values.

[0078] Specifically, when the data is in a stable state, enter the real-time sensitive state, initialize a timer, and increment the time value per frame. If the cumulative time value is less than or equal to 2 seconds, directly assign the current position data to the average position data and the current rotation data to the average rotation data. When the time value exceeds 2 seconds, reset the time value to 0 and close the real-time sensitive state.

[0079] Specifically, when the data tends to be in a jumping state, enter the smooth tracking state for the first time. At this time, initialize the index and fill the historical position array smoothpos1 with the current position data, and fill the historical rotation array smoothrot1 with the current rotation data. In each subsequent frame, put the new position data into the specified index of the historical position array, then increment the index and ensure it is always within the valid range (reusing the historical position array in a loop). Put the new rotation data into the specified index of the historical rotation array, then increment the index and ensure it is always within the valid range (reusing the historical rotation array in a loop).

[0080] Traverse all the position data in the historical position array smoothpos1 and accumulate them, and finally divide by the array length to get the average position data. Use an iterative method to gradually fuse and average the quaternions until there is only one rotation data left in the array, and the remaining one rotation data is the average rotation data.

[0081] S103. Control the first virtual human model to perform synchronous motion based on the motion data, and control the first virtual human model to strike the virtual prop.

[0082] The first virtual human model is the virtual human model corresponding to the training user in the virtual training scenario. The motion state of the first virtual human model in the virtual training scenario should be consistent with the running state of the training user in the actual scenario.

[0083] Through the IK binding technology and the animation synchronization mechanism, map the motion data collected in S102 to the first virtual human model, and drive the bone joints in real time to generate animations, so as to realize the synchronous motion of the first virtual human model and the training user.

[0084] Controlling the first virtual human model to perform synchronous motion based on action data mainly includes two steps: binding the first virtual human model with the action and controlling the first virtual human model.

[0085] First, the IK animation binding of the first virtual human model mainly includes: when creating a human model in 3D modeling software, creating IK (Inverse Kinematics, abbreviated as IK) controllers for its skeletal system, including the head, shoulder joints, spine, limbs, and the joint skeletons of hands and feet. After importing into Unity, bind the 3D human model with the skeleton, determine which joints each vertex is affected by, and ensure smooth deformation of the model surface when the skeleton moves through weight assignment.

[0086] Second, the control process of the first virtual human model mainly includes: the human model driving module updates the character animation skeleton in the scene in real time according to the above action data in each frame. First, check whether the device identifier is consistent with the first virtual human model. If it is consistent, then start to update the bone transformation information in the scene, and then traverse all the bone transformation components of the human model, assign corresponding type data according to the bone type, and obtain its position, rotation, and scaling information in the scene space.

[0087] In a possible implementation, controlling the first virtual human model to perform synchronous motion based on action data includes: traversing the bone components of the first virtual human model; obtaining the action data of the current bone component and the action data of the parent bone; constructing a first quaternion based on the inverse operation of the action data of the parent bone, the action data of the current bone, and the action data of the parent bone; performing Euler processing on the first quaternion to obtain a second quaternion; determining the target action data of the current bone based on the second quaternion and the action data of the current bone.

[0088] Before performing bone assignment driving, it is necessary to first calculate a quaternion (defined as usedQ), which represents the rotation of the current bone relative to its parent bone after transformation. This relative transformation is achieved by multiplying the inverse of the original parent bone rotation (defined as originalParentRot) by the current bone rotation (CurrentRot), and then multiplying by the original parent bone rotation itself. Convert usedQ into Euler angle form to get transedRot, and then construct a new quaternion (defined as finalBoneQ) with the Euler angles and multiply it by the rotation of the bone itself (defined as originalBoneRot). This step mainly considers ensuring the original initial state of the bone node and its relative rotation in the space of its parent bone. Similarly, the position information is processed in a similar way to ensure that the position of the bone node is correctly located at the final position in the coordinate system of its parent bone.

[0089] The abstract calculation formula described above is as follows:

[0090] 1. Calculate the first quaternion group usedQ (representing the rotation after transformation relative to the parent bone): usedQ = Inverse(originalParentRot) * CurrentRot * originalParentRot.

[0091] 2. Convert the first quaternion usedQ to Euler angle form to obtain transedRot: transedRot = EulerAngles(usedQ).

[0092] 3. Construct the final local rotation quaternion, i.e., the second quaternion finalBoneQ: finalBoneQ = Quaternion.Euler(transedRot) * originalBoneRot.

[0093] So far, the full - body driving of the first virtual human model is completed in this embodiment.

[0094] In a possible implementation, when synchronizing the actions of the first virtual human model with the training user, control the first virtual human model to hit a virtual prop. In this embodiment, the virtual prop is taken as an example of a virtual tennis ball for illustration.

[0095] The movement principle of the virtual tennis ball is based on the physical engine principle. Increase the racket - hitting force of the first virtual human model to control the virtual tennis ball to simulate the movement trajectory of an actual tennis ball. As Figure 2 shown, the control of the virtual tennis ball mainly includes two parts. First, the physical properties of the virtual tennis ball. Second, the first hitting force applied to the virtual tennis ball.

[0096] First, setting the physical properties of the virtual tennis ball: Add physical properties to the tennis ball to make its movement natural. This property is assigned through a configuration file and can be dynamically modified.

[0097] Create a Rigidbody component: Add a Rigidbody component to the virtual tennis ball object to make it controlled by the physical engine. Select "Dynamic" as the type of the Rigidbody because the virtual tennis ball is a dynamically moving object.

[0098] Set the mass: In the rigidbody component, adjust the "Mass" property to simulate the mass of the tennis ball. It is about 50 grams.

[0099] Initial velocity: Set the initial velocity of the tennis ball through the rigidbody.velocity property, e.g., rigidbody.velocity = new Vector3(xSpeed, ySpeed, zSpeed); to simulate the velocity when serving or hitting the ball.

[0100] Angular Velocity: Simulate the spin of the tennis ball by setting the "AngularVelocity" of the Rigidbody. For example: rigidbody.angularVelocity = new Vector3(xSpin, ySpin, zSpin); where the values on the x, y, and z axes represent the rotational speeds around each axis.

[0101] Drag & Angular Drag: In the Rigidbody component, adjust the "Drag" property to simulate the effect of air resistance on the linear motion of the tennis ball. The higher the value, the greater the resistance and the faster the tennis ball decelerates. Similarly, "AngularDrag" is used to simulate the effect of air resistance on rotation.

[0102] Bounciness: Use the "Bounciness" and "Friction" in the "Material" property to simulate the bounciness of the tennis ball. The higher the Bounciness value, the higher the tennis ball bounces; Friction affects the sliding degree when the tennis ball contacts the ground or other objects.

[0103] Collision Detection and Bounce Behavior: Set the collider. Use SphereCollider to enclose the tennis ball model and adjust its size to be the same as the tennis ball.

[0104] Second, the first hitting force applied to the virtual tennis ball: According to the interaction logic of rehabilitation training, the force feedback added to the tennis ball separately. Mainly when certain conditions are met (triggered by the game rule module), according to the patient's racket swing angle, direction, and acceleration, give the tennis ball a feedback force to form a hitting force when the tennis ball is not physically struck. This can meet the simple hitting effect of patients who cannot go to the tennis ball landing point or are not familiar with tennis skills, ensure the ball can cross the net, and reduce the difficulty of rehabilitation training.

[0105] In a possible implementation, controlling the first virtual human model to strike the virtual prop includes: when a striking rule is triggered, calculating the swing angle, swing direction, and acceleration of the first virtual human model based on the action data; determining the first striking force of the virtual prop based on the swing angle, swing direction, and acceleration; and pre-adding the first striking force to the virtual prop with basic physical properties, where the basic physical properties include one or more of the following: the mass of the virtual prop, the rotation angle of the virtual prop, the air resistance of the virtual prop, the elasticity of the virtual prop, and the collision rules corresponding to the virtual prop.

[0106] The principle is to calculate the force (power) and direction of the patient's arm sliding, and give a striking feedback force to the tennis ball through this method.

[0107] Calculate the sliding direction vector, based on the difference between the current position and the previous position, which reflects the sliding direction of the patient's hand in this frame. Calculate the basic force based on the change in the hand rotation angle and update the rotation state of the previous frame. Record and accumulate the angle changes of multiple consecutive frames and add them to the sliding direction vector.

[0108] If the sliding distance is too small, ignore this sliding and clear the historical cache.

[0109] Perform corresponding amplification or reduction processing on each component of the sliding direction vector to modify the game difficulty. For the case of the arm sliding backward, ignore the force generated. Finally, return the sliding force value after a series of adjustments, that is, the striking feedback force of the tennis ball, so that the tennis ball can cross the net in the reverse direction. This method is detected when hitting is allowed in the game rule module, enabling the patient to swing the arm to hit the tennis ball over the net without having to use the racket to hit the tennis ball and move.

[0110] Further, after applying the first striking force to the virtual tennis ball, the virtual tennis ball moves in the virtual training scenario. The following introduces the prediction method for the moving trajectory points of the virtual tennis ball.

[0111] First, create an array results of size step according to the input parameters to store each point on the predicted trajectory. Set a fixed time step (timestep) of 0.1 second, and calculate the gravity acceleration increment gravityAccel within each time step based on this time step and the global physical gravity. Calculate the moving distance moveStep of the first step according to the initial velocity velocity.

[0112] Iterative calculation: For each time step from 0 to step-1: a. Add the current gravity acceleration increment gravityAccel to moveStep to simulate the acceleration change and cumulative displacement of the object due to gravity in each time step. b. Update the current position startPos to the position of the previous step plus the new moveStep. c. Store the updated position startPos at the corresponding index in the results array results.

[0113] Collect trajectory points: After the loop ends, the results array will contain a series of position vectors that represent the expected position of the object after timestep seconds, starting from the given initial position and velocity, under the influence of gravity and without considering other external forces.

[0114] Return results: Finally, the filled results array is returned. By connecting these points, the parabolic trajectory of the object within the specified time interval can be drawn.

[0115] In one possible implementation, to increase the fun of training, a second virtual human model is presented in the virtual training scene. This second virtual human model is a pre-defined virtual human model in the virtual training scene. Its primary function is to serve as a virtual opponent for the training user, achieving a tennis sparring effect and enhancing the fun of the training process.

[0116] Specifically, when it is detected that the training user triggers the hitting rule, the current position of the virtual prop and the basic physical properties of the virtual prop are obtained; based on the current position and the basic physical properties, the predicted trajectory point of the virtual prop is determined, wherein the predicted trajectory point includes the landing position; the second virtual human body model is controlled to move toward the landing position; when the distance between the position of the second virtual human body model and the landing position is less than a set distance, a second hitting force is calculated based on the basic physical properties of the virtual prop; and the second hitting force is applied to the virtual prop to control the virtual prop to move toward the direction of the training user.

[0117] like Figure 3 As shown, it is determined whether the landing position in the predicted trajectory point has not passed the net or is out of bounds. If the landing position has not passed the net or is out of bounds, the second virtual human model remains in place and does not move. If the virtual tennis ball reaches the landing position and the landing position is within the range set by the rules, but the second virtual human model does not move within the preset range of the landing position, it is determined that the second virtual human model has missed the shot and cannot return the ball.

[0118] Calculate the distance between the virtual tennis ball and the second virtual human model based on their positions. If the distance is less than or equal to 1 meter, the second virtual human model can attempt to hit the ball.

[0119] Utilize Newton's theorem to calculate the hitting force of the tennis racket to hit the tennis ball towards the target position. Specifically, calculating the second hitting force based on the basic physical properties of the virtual prop includes: obtaining the gravity vector of the virtual prop; calculating the velocity vector of the virtual prop in the vertical direction and the velocity vector in the horizontal direction based on the gravity vector; adding the velocity vector in the vertical direction and the velocity vector in the horizontal direction to obtain the second hitting force.

[0120] Obtain the gravity vector in the game world: gravity = Physics.gravity;

[0121] Calculate the initial velocity in the vertical direction (upward motion): According to the free - fall formula, the time required for an object to reach the highest point is half of the total flight time. Therefore, the initial upward velocity can be calculated by the following formula: upVelocity = -0.5 * gravity * flyTime;

[0122] Calculate the initial velocity in the horizontal direction (uniform linear motion): Assume that the object moves to the target position at a constant speed in the horizontal direction. Then the initial velocity can be obtained by the following formula: forwardVelocity = (target - startPoint) / flyTime;

[0123] Synthesize the velocity vector: Add the vertical and horizontal velocity vectors to obtain the overall initial velocity vector: result = upVelocity + forwardVelocity; Move the racket to the ideal position: Calculate the ideal position that the racket should reach according to the flying direction of the tennis ball, and set its y - coordinate to the y - coordinate of the racket root to keep it moving on the horizontal plane. Then set a random variable (-1, 1) to control the return direction and return force of the tennis ball.

[0124] Apply the reaction force of the hitting force of the tennis ball to hit the ball towards the patient side to complete one round.

[0125] Train the user to trigger the hitting rules, including the following three methods.

[0126] First, standard serve: After training the user's non - racket - holding hand moves upward beyond the shoulder, within one second, the racket - holding arm swings to make a serving motion. The game rule module automatically generates a tennis ball and applies the user's racket - swinging force to hit the tennis ball towards the second virtual human model side. This method is suitable for patients who are familiar with tennis and have good upper - limb joint movement.

[0127] Second, raising the hand to serve: When the user raises any arm above the head for more than 2 seconds, it is determined that the patient serves. The game rule module automatically applies a serving force to the tennis ball and hits the tennis ball towards the second virtual human model side. This method is applicable to patients with general upper limb joint movements but who are not familiar with the tennis mechanism.

[0128] Third, automatic serving. After the start of the game or when the ball goes out of bounds, if the training user does not make any serving actions within 5 seconds, the system automatically applies a serving force to the tennis ball and hits the tennis ball towards the second virtual human model side. This method is applicable to patients with upper limbs that cannot be raised high but can make small-amplitude movements.

[0129] In another possible implementation, the training user may have lower limb dysfunction and cannot move, adding two other movement methods.

[0130] First, free movement: The training user is allowed to move freely in the virtual training scenario. When the distance between the training user's hand and the tennis ball is less than 1 meter, the recording of the patient's hitting feedback force is started. When the hitting feedback is satisfied, the virtual tennis ball is hit towards the opponent.

[0131] Second, fixed position: The patient does not need to move. When the ball enters the training user's area and bounces after landing, the recording of the patient's hitting feedback force is started. At this time, when the hitting feedback is satisfied, the tennis ball can also be hit towards the opponent.

[0132] In this embodiment, three serving methods are provided for patients with different abilities. At the same time, taking into account patients with lower limb dysfunction, two hitting timing determination strategies of free movement and fixed position are provided, and the racket can be switched to change the holding hand. To provide specific training methods for different users.

[0133] S104. During the rehabilitation training process, analyze and statistically process the action data of the training user to obtain a rehabilitation evaluation result.

[0134] Among them, the action data of the training user is statistically analyzed to obtain the score of the training user and the score of the second virtual human model.

[0135] If after the first virtual human model hits the ball and the virtual tennis ball exceeds the boundary line of the virtual training scenario, then it is determined that the second virtual human model scores. If after the second virtual human model hits the ball and the virtual tennis ball exceeds the boundary line of the virtual training scenario, then it is determined that the first virtual human model scores.

[0136] If after the first virtual human model hits the ball and the virtual tennis ball does not exceed the boundary line of the virtual training scenario, and the second virtual human model fails to hit the ball successfully, then it is determined that the first virtual human model scores. If after the second virtual human model hits the ball and the virtual tennis ball does not exceed the boundary line of the virtual training scenario, and the first virtual human model fails to hit the ball successfully, then it is determined that the second virtual human model scores.

[0137] Furthermore, it is also possible to record the game time, the number of successful hits by the left hand, the number of successful hits by the right hand, the number of hitting mistakes, the number of serves, and the total movement distance data of both hands. Specifically as follows:

[0138] Scores of both sides: the score of the training user and the score of the second virtual human model.

[0139] Total movement distance: Record the distances that the left and right hands move during all times from the start to the end of the game respectively, with the unit in meters. For example, the total movement distance of the left hand: 100 meters.

[0140] Number of serves: Record the number of serves using various serving methods.

[0141] Number of successful hits: Record the number of times the left and right hands successfully trigger the hitting feedback force of the tennis ball respectively. This data is used as a parameter to evaluate the upper limb movement degree of the patient, and it does not distinguish whether it passes the net or hits the net, etc.

[0142] Number of missed hits: Record the number of times the left and right hands fail to trigger the hitting feedback force respectively.

[0143] Game time: Record the total training time from the start to the end of the training.

[0144] Based on the data obtained from the above statistics, each training data is recorded separately. A line chart is drawn through a visual data graph plugin and the line chart is displayed. The specific implementation is as follows:

[0145] Create multiple data lists and arrays for the above data. Each list represents the values of a statistical dimension in different training sessions. Create a new graph plugin in Unity to display the line chart. For each data type to be displayed, configure a data series. For example, create two series for the total movement distances of the left and right hands respectively, set the data source of each series to its corresponding data array, and specify the values of the X-axis (training number and date) and the Y-axis (specific statistical data).

[0146] Load the chart data that needs to be updated in real time through a script and display it in the user interface display module, as Figure 4 shown.

[0147] Furthermore, as a virtual reality training game, the user interface display module uses a 3D stereoscopic UI interface for display. Use the built-in UI plugin in Unity to display the UI data in a three-dimensional manner, which is suitable for virtual reality. Display the above line chart in the virtual training scene to display information such as the game process, score statistics, and patient movement data in real time.

[0148] An embodiment of the present disclosure provides a rehabilitation training evaluation method based on virtual reality. The method includes: presenting a virtual training scene corresponding to a training user, where a virtual prop and a first virtual human model are presented in the virtual training scene; obtaining the motion data of the training user; controlling the first virtual human model to perform synchronous motion based on the motion data, and controlling the first virtual human model to strike the virtual prop; and during the rehabilitation training process, analyzing and statistically processing the motion data of the training user to obtain a rehabilitation training result. Since during the process of the training user performing rehabilitation training in the virtual training scene, the motion data of the training user can be collected, and based on the motion data, the first virtual human model in the virtual training scene is controlled to perform synchronous actions, and when performing synchronous actions, the virtual prop is struck, it helps the patient to perform efficient rehabilitation training that conforms to the motion law of the actual prop in the virtual reality environment to meet personalized needs and improve the fun of training. And the key training data during the rehabilitation training process is statistically processed, and the rehabilitation situation is evaluated based on the key training data, so as to provide data support for formulating a personalized training method for the user in the future.

[0149] Figure 5 FIG. is a schematic structural diagram of a rehabilitation training evaluation device based on virtual reality in an embodiment of the present disclosure, as Figure 5 shown, the rehabilitation training evaluation device 50 provided in the embodiment of the present disclosure mainly includes: a virtual training scene display module 51, configured to present a virtual training scene corresponding to a training user, where a virtual prop and a first virtual human model are presented in the virtual training scene; a motion data acquisition module 52, configured to obtain the motion data of the training user; a first model control module 53, configured to control the first virtual human model to perform synchronous motion based on the motion data, and control the first virtual human model to strike the virtual prop; and a rehabilitation evaluation module 54, configured to analyze and statistically process the motion data of the training user during the rehabilitation training process to obtain a rehabilitation evaluation result.

[0150] In a possible implementation manner, the motion data acquisition module 52 is configured to, for the motion data of any joint point, determine whether the difference between the current motion data and the average motion data is greater than a set threshold; when the difference is less than or equal to the set threshold, enter the real-time sensitivity state; in the real-time sensitivity state, initialize a timer, and increase the time value in units of each frame; when the time value of the timer is less than or equal to the set time value, assign the obtained current motion data to the average motion data; when the time value of the timer is greater than the set time value, reset the timer and exit the real-time sensitivity state; when the difference is greater than the set threshold, enter the smooth tracking state; in the smooth tracking state, initialize an index, and fill the current motion data into the historical data array; calculate the average motion data from all the motion data in the historical data array.

[0151] In a possible implementation, the first model control module 53 is specifically configured to traverse the bone components of the first virtual human model; obtain the motion data of the current bone component and the motion data of the parent bone; construct a first quaternion based on the inverse operation of the motion data of the parent bone, the motion data of the current bone, and the motion data of the parent bone; perform Euler processing on the first quaternion to obtain a second quaternion; and determine the target motion data of the current bone based on the second quaternion and the motion data of the current bone.

[0152] In a possible implementation, the first model control module 53 is specifically configured to, when a hitting rule is triggered, calculate the swing angle, swing direction, and acceleration of the first virtual human model based on the motion data; determine the first hitting force of the virtual prop based on the swing angle, swing direction, and acceleration; and pre-add the first hitting force to the virtual prop with basic physical properties, where the basic physical properties include one or more of the following: the mass of the virtual prop, the rotation angle of the virtual prop, the air resistance of the virtual prop, the elasticity of the virtual prop, and the collision rule corresponding to the virtual prop.

[0153] In a possible implementation, a second virtual human model is further presented in the virtual training scenario. The second virtual human model is a virtual human model preset in the virtual training scenario, and further includes: a second model control module, configured to, when it is detected that the training user triggers a hitting rule, obtain the current position of the virtual prop and the basic physical properties of the virtual prop; determine the predicted trajectory points of the virtual prop based on the current position and the basic physical properties, where the predicted trajectory points include the landing position; control the second virtual human model to move towards the landing position; and when the distance between the position of the second virtual human model and the landing position is less than a set distance, calculate a second hitting force based on the basic physical properties of the virtual prop; and apply the second hitting force to the virtual prop to control the virtual prop to move in the direction of the training user.

[0154] In a possible implementation, the second model control module is specifically configured to obtain the gravity vector of the virtual prop; calculate the velocity vector of the virtual prop in the vertical direction and the velocity vector in the horizontal direction based on the gravity vector; and add the velocity vector in the vertical direction and the velocity vector in the horizontal direction to obtain the second hitting force.

[0155] In a possible implementation, the second model control module is specifically configured to calculate the increment of gravitational acceleration within each step based on a preset step size and the global physical gravity; for each step, calculate the moving distance of the current step; add the increment of gravitational acceleration within the current step to the moving distance of the current step to obtain the accumulated displacement of acceleration; and add the position data of the previous step to the accumulated displacement of acceleration to obtain the predicted trajectory point of the current step.

[0156] The rehabilitation training evaluation device based on virtual reality provided by the embodiments of the present disclosure can execute the steps performed in the rehabilitation training evaluation method based on virtual reality provided by the method embodiments of the present disclosure, and the implementation steps and beneficial effects are not described herein again.

[0157] Figure 6 It is a schematic structural diagram of an electronic device in the embodiments of the present disclosure. This electronic device can be a rehabilitation training evaluation device based on virtual reality. Specifically, refer to Figure 6 which shows a schematic structural diagram of an electronic device 600 suitable for implementing the embodiments of the present disclosure.

[0158] As Figure 6 shown, the electronic device 600 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 601, which can execute various appropriate actions and processes according to the program stored in the read-only memory (ROM) 602 or the program loaded from the storage device 608 into the random access memory (RAM) 603 to implement the rehabilitation training evaluation method based on virtual reality in the embodiments described in the present disclosure. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. The input / output (I / O) interface 605 is also connected to the bus 604.

[0159] Generally, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 can allow the electronic device 600 to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 6 shows the electronic device 600 having various devices, it should be understood that it is not required to implement or include all the shown devices. Instead, more or fewer devices may be implemented or included.

[0160] In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium. The computer program contains program codes for executing the methods shown in the flowcharts, so as to implement the rehabilitation training evaluation method based on virtual reality as described above. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are executed.

[0161] It should be noted that the above-mentioned computer-readable medium in the present disclosure can be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, in which computer-readable program codes are carried. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and the computer-readable signal medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program codes contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0162] In some embodiments, the client and the server can communicate using any currently known or future-developed network protocol such as HTTP (HyperText Transfer Protocol), and can be interconnected with digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0163] The above computer-readable medium can be included in the above electronic device; it can also exist separately and not be assembled into the electronic device.

[0164] In a possible embodiment of the present disclosure, the above computer-readable medium carries one or more programs, and when the one or more programs are executed by the terminal device, the terminal device can implement the virtual reality-based rehabilitation training evaluation method described in any one of the above embodiments.

[0165] In a possible embodiment of the present disclosure, when the one or more programs are executed by the terminal device, the terminal device can also perform the other steps described in the above embodiments.

[0166] Computer program code for performing the operations of the present disclosure can be written in one or more programming languages or combinations thereof. The above programming languages include, but are not limited to, object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., by using an Internet service provider to connect through the Internet).

[0167] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur in a different order than that noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0168] The units described in the embodiments of the present disclosure can be implemented in software or in hardware. In some cases, the name of the unit does not constitute a limitation on the unit itself.

[0169] The functions described above herein can be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that can be used include: Field Programmable Gate Arrays (FPGA), Application Specific Integrated Circuits (ASIC), Application Specific Standard Products (ASSP), System on a Chip (SOC), Complex Programmable Logic Devices (CPLD), and the like.

[0170] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a Read-Only Memory (ROM), an Erasable Programmable Read-Only Memory (EPROM or Flash memory), an optical fiber, a portable Compact Disc Read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0171] The above description is only a preferred embodiment of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept. For example, the technical solutions formed by mutually replacing the above features with the technical features (but not limited to) having similar functions disclosed in the present disclosure.

[0172] In addition, although the operations are depicted in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in sequential order. In certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented separately or in any suitable sub-combination in multiple embodiments.

[0173] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. On the contrary, the specific features and acts described above are merely example forms for implementing the claims.

Claims

1. A rehabilitation training evaluation method based on virtual reality, characterized in that Including: Displaying a virtual training scenario corresponding to the training user, where a virtual prop and a first virtual human model are presented in the virtual training scenario; Collecting the motion data of the training user; Based on the motion data, controlling the first virtual human model to perform synchronous motion, and controlling the first virtual human model to strike the virtual prop; During the rehabilitation training process, analyzing and statistically processing the motion data of the training user to obtain a rehabilitation evaluation result; Among them, for the collection of the motion data of the training user, the motion data includes rotation data and position data, including: for the motion data of any joint point, comparing whether the square difference between the average position data and the current position data exceeds a first preset threshold, and whether the angle difference between the average rotation data and the current rotation exceeds a second preset threshold; if neither exceeds, enter the real-time sensitivity state; in the real-time sensitivity state, initialize the timer, and increase the time value in units of each frame; when the time value of the timer is less than or equal to the set time value, assign the obtained current position data to the average position data; assign the obtained current rotation data to the average rotation data; when the time value of the timer is greater than the set time value, reset the timer and exit the real-time sensitivity state; when any one exceeds, enter the smooth tracking state; in the smooth tracking state, initialize the index, fill the current position data into the historical position array, and fill the current rotation data into the historical rotation array; traverse all the position data in the historical position array and accumulate them, and finally divide by the array length to obtain the average position data; use an iterative method to gradually fuse and average the quaternions until there is only one rotation data left in the array, and the remaining one rotation data is the average rotation data; Among them, the controlling the first virtual human model to perform synchronous motion based on the motion data includes: traversing the bone components of the first virtual human model; obtaining the motion data of the current bone component and the motion data of the parent bone; multiplying the inverse operation of the motion data of the parent bone, the motion data of the current bone, and the motion data of the parent bone to construct a first quaternion; performing Euler processing on the first quaternion to obtain a second quaternion; multiplying the second quaternion by the bone's own motion data to obtain the target motion data of the current bone.

2. The method according to claim 1, characterized in that, The controlling the first virtual human model to strike the virtual prop includes: When the hitting rule is triggered, calculating the swing angle, swing direction, and acceleration of the first virtual human model based on the motion data; Determining the first hitting force of the virtual prop based on the swing angle, swing direction, and acceleration; Pre-adding the first hitting force to the virtual prop with basic physical properties, where the basic physical properties include one or more of the following: the mass of the virtual prop, the rotation angle of the virtual prop, the air resistance of the virtual prop, the elasticity of the virtual prop, the collision rules corresponding to the virtual prop.

3. The method according to claim 1, characterized in that, A second virtual human model is also presented in the virtual training scenario. The second virtual human model is a virtual human model preset in the virtual training scenario. The method further includes: When it is detected that the training user triggers a hitting rule, obtaining the current position of the virtual prop and the basic physical properties of the virtual prop; Determining the predicted trajectory points of the virtual prop based on the current position and the basic physical properties, wherein the predicted trajectory points include the landing position; Controlling the second virtual human model to move towards the landing position; When the distance between the position of the second virtual human model and the landing position is less than a set distance, calculating a second hitting force based on the basic physical properties of the virtual prop; Applying the second hitting force to the virtual prop to control the virtual prop to move in the direction of the training user.

4. The method according to claim 3, wherein The calculating the second hitting force based on the basic physical properties of the virtual prop includes: Obtaining the gravity vector of the virtual prop; Calculating the velocity vector of the virtual prop in the vertical direction and the velocity vector in the horizontal direction based on the gravity vector; Adding the velocity vector in the vertical direction and the velocity vector in the horizontal direction to obtain the second hitting force.

5. The method according to claim 3, wherein The determining the predicted trajectory points of the virtual prop based on the current position and the basic physical properties of the virtual prop includes: Calculating the increment of gravitational acceleration within each step length based on a preset step length and the global physical gravity; Calculating the moving distance of the current step length for each step length; Accumulating the increment of gravitational acceleration within the current step length to the moving distance of the current step length to obtain the accumulated displacement of acceleration; Adding the position data of the previous step to the accumulated displacement of acceleration to obtain the predicted trajectory point of the current step.

6. A rehabilitation training evaluation device based on virtual reality, characterized in that, including: A virtual training scenario display module for displaying the virtual training scenario corresponding to the training user, where a virtual prop and a first virtual human model are presented in the virtual training scenario; An action data acquisition module for acquiring the action data of the training user; A first model control module for controlling the first virtual human model to perform synchronous movement based on the action data and controlling the first virtual human model to hit the virtual prop; A rehabilitation evaluation module for analyzing and statistically processing the action data of the training user during the rehabilitation training process to obtain a rehabilitation evaluation result; Among them, collecting the action data of the training user, where the action data includes rotation data and position data, includes: for the action data of any joint point, comparing whether the squared difference between the average position data and the current position data exceeds a first preset threshold, and whether the angular difference between the average rotation data and the current rotation exceeds a second preset threshold; if neither exceeds, enter the real-time sensitive state; in the real-time sensitive state, initialize a timer and increment the time value in units of each frame; when the time value of the timer is less than or equal to the set time value, assign the obtained current position data to the average position data; assign the obtained current rotation data to the average rotation data; when the time value of the timer is greater than the set time value, reset the timer and exit the real-time sensitive state; when any one exceeds, enter the smooth tracking state; in the smooth tracking state, initialize an index, fill the current position data into the historical position array, and fill the current rotation data into the historical rotation array; traverse all the position data in the historical position array and accumulate them, and finally divide by the array length to obtain the average position data; use an iterative method to gradually fuse and average the quaternions until there is only one rotation data left in the array, and the remaining one rotation data is the average rotation data. Among them, controlling the first virtual human model to perform synchronous motion based on the action data includes: traversing the bone components of the first virtual human model; obtaining the action data of the current bone component and the action data of the parent bone; multiplying the inverse operation of the action data of the parent bone, the action data of the current bone, and the action data of the parent bone to construct a first quaternion; performing Euler processing on the first quaternion to obtain a second quaternion; multiplying the second quaternion by the action data of the bone itself to obtain the target action data of the current bone.

7. An electronic device, characterized in that, The electronic device includes: One or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the virtual reality-based rehabilitation training evaluation method according to any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the virtual reality-based rehabilitation training evaluation method according to any one of claims 1-5.

Images