Multi-dimensional motion function evaluation method and device, computer equipment and medium

Abstract

The embodiment of the invention provides a multi-dimensional motion function evaluation method and device, computer equipment and a medium, and relates to the technical field of artificial intelligence, and the method comprises the following steps: respectively collecting image data of a target object in a plurality of azimuth angles in a motion process; acquiring a skeleton coordinate sequence of the target object at each azimuth angle according to the image data; according to the skeleton coordinate sequence, obtaining a first dimension feature at each azimuth angle; calculating the sum of the scores corresponding to the first dimension features according to the corresponding relationship between the first dimension features and the scores, and taking the sum as a third dimension feature; and determining an evaluation result of the motion function of the target object according to the second dimension feature and the third dimension feature. According to the scheme, through the multi-angle image data and the multi-dimensional evaluation method, the accuracy of exercise function evaluation is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and particularly relates to an evaluation method, device, computer device and medium for multi-dimensional motion functions. Background Art

[0002] At present, the problem of population aging in China is becoming increasingly serious, and hemiplegia is a high-incidence disease among the elderly population. Therefore, the rehabilitation treatment for the elderly after hemiplegia is particularly important. Due to the damage of the nervous system, hemiplegic patients lack the ability of muscle movement control, especially the ability of movement coordination. Therefore, the rehabilitation of hemiplegic patients requires a long-term and comprehensive training. Therefore, objective and accurate evaluation results of motor function can provide an effective basis for formulating an efficient and reasonable rehabilitation training plan. Therefore, it is of great significance to comprehensively evaluate the motor ability of hemiplegic patients.

[0003] Currently, the commonly used methods for evaluating the motor function of stroke in clinical practice are rating scales such as the Fugl-Meyer motor function evaluation method. Most of these traditional evaluation methods rely on the observation and manual operation of therapists, and the adjustment of the human rehabilitation state and training parameters depends on the subjective judgment of rehabilitation therapists, lacking a unified quantitative standard, thus affecting the accuracy of rehabilitation training.

[0004] In response to this problem, researchers at home and abroad have proposed to establish a quantitative evaluation system for rehabilitation training. Currently, the evaluation methods of human motor ability mainly focus on the calculation and classification of single indicators such as joint angle, joint position, and joint movement speed, and it is impossible to see deeper patterns from these features. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide an evaluation method for multi-dimensional motion functions to solve the technical problem in the prior art that the motor ability is mainly evaluated by single indicators such as joint angle, joint position, and joint movement speed. The method includes:

[0006] During the movement of the target object, image data of the target object are respectively collected at multiple azimuth angles;

[0007] According to the image data, a sequence of bone coordinates of the target object at each azimuth angle is obtained;

[0008] According to the sequence of bone coordinates at each azimuth angle, first-dimensional features at each azimuth angle are obtained, where the first-dimensional features include multiple joint movement-related features and relative position features;

[0009] According to the first-dimensional features, second-dimensional features of the target object are judged, where the second-dimensional features include multiple evaluation index features related to motor ability;

[0010] Calculate the sum of the scores corresponding to each first - dimension feature according to the correspondence between the first - dimension feature and the score, and use the sum as the third - dimension feature.

[0011] Determine the evaluation result of the target object's motor function according to the second - dimension feature and the third - dimension feature.

[0012] The embodiment of the present invention also provides an evaluation device for multi - dimensional motor function to solve the technical problem in the prior art that the motor ability is mainly evaluated by single indicators such as joint angle, joint position, joint movement speed, etc. The device includes:

[0013] A data acquisition module, configured to respectively acquire the image data of the target object at multiple azimuth angles during the movement of the target object.

[0014] An acquisition module for skeletal coordinate sequences, configured to obtain the skeletal coordinate sequences of the target object at each azimuth angle according to the image data.

[0015] A first - dimension feature calculation module, configured to obtain the first - dimension features at each azimuth angle according to the skeletal coordinate sequences at each azimuth angle, where the first - dimension features include multiple joint - movement - related features and relative - position features.

[0016] A second - dimension feature judgment module, configured to judge the second - dimension feature of the target object according to the first - dimension feature, where the second - dimension features include multiple evaluation - index features related to motor ability.

[0017] A third - dimension feature calculation module, configured to calculate the sum of the scores corresponding to each first - dimension feature according to the correspondence between the first - dimension feature and the score, and use the sum as the third - dimension feature.

[0018] An evaluation report generation module, configured to determine the evaluation result of the target object's motor function according to the second - dimension feature and the third - dimension feature.

[0019] The embodiment of the present invention also provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the above - mentioned arbitrary evaluation method for multi - dimensional motor function to solve the technical problem in the prior art that the motor ability is mainly evaluated by single indicators such as joint angle, joint position, joint movement speed, etc.

[0020] The embodiment of the present invention also provides a computer - readable storage medium, which stores a computer program for executing the above - mentioned arbitrary evaluation method for multi - dimensional motor function to solve the technical problem in the prior art that the motor ability is mainly evaluated by single indicators such as joint angle, joint position, joint movement speed, etc.

[0021] Compared with the prior art, the beneficial effects that can be achieved by at least one of the above technical solutions adopted in the embodiments of this specification at least include:

[0022] Collect image data of the target object from multiple angles, select videos from different angles when calculating different features, so that the action information of the patient can be compared from multiple angles, making the evaluation result more accurate; evaluate the motor function through multiple dimensions of the first dimension feature, the second dimension feature and the third dimension feature, and the obtained evaluation result is three-dimensional and highly accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0024] Figure 1 is a flowchart of a method for evaluating multi-dimensional motor function provided by an embodiment of the present invention;

[0025] Figure 2 is a schematic diagram of multi-dimensional motion characteristics for implementing the above method for evaluating multi-dimensional motor function provided by an embodiment of the present invention;

[0026] Figure 3 is a schematic diagram of human joint points for implementing the above method for evaluating multi-dimensional motor function provided by an embodiment of the present invention;

[0027] Figure 4 is a schematic diagram of spinal measurement for implementing the above method for evaluating multi-dimensional motor function provided by an embodiment of the present invention;

[0028] Figure 5 is a schematic diagram of joint range of motion calculation for implementing the above method for evaluating multi-dimensional motor function provided by an embodiment of the present invention;

[0029] Figure 6 is a block diagram of the structure of a computer device provided by an embodiment of the present invention;

[0030] Figure 7 is a block diagram of the structure of an apparatus for evaluating multi-dimensional motor function provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0031] The embodiments of the present application will be described in detail below with reference to the drawings.

[0032] The following describes the implementation manners of the present application through specific examples. Those skilled in the art can easily understand other advantages and effects of the present application from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The present application can also be implemented or applied through other different specific implementation manners. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that, without conflict, the following embodiments and the features in the embodiments can be combined with each other. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without making creative efforts belong to the scope protected by the present application.

[0033] In an embodiment of the present invention, a method for evaluating multi-dimensional motion functions is provided. As Figure 1 and Figure 2 shown, the method includes:

[0034] Step S101: During the movement of the target object, image data of the target object is collected at multiple azimuth angles respectively;

[0035] Step S102: According to the image data, a sequence of skeletal coordinates of the target object at each azimuth angle is obtained;

[0036] Step S103: According to the sequences of skeletal coordinates at each azimuth angle, first-dimensional features at each azimuth angle are obtained, where the first-dimensional features include multiple joint motion-related features and relative position features;

[0037] Step S104: According to the first-dimensional features, the second-dimensional features of the target object are judged, where the second-dimensional features include multiple evaluation index features related to motion ability;

[0038] Step S105: According to the correspondence between the first-dimensional features and scores, the sum of the scores corresponding to each first-dimensional feature is calculated, and the sum is used as the third-dimensional feature;

[0039] Step S106: According to the second-dimensional features and the third-dimensional features, the evaluation result of the motion function of the target object is determined.

[0040] Specifically, the image data is converted into joint point space coordinate data, and a sequence of skeletal coordinates of the moving object at each azimuth is generated according to the joint point space coordinate data, where the azimuth includes the front and the side;

[0041] As Figure 3 、 Figure 4 、 Figure 5As shown, the first - dimension features and the first - dimension additional features in each orientation are calculated based on the skeletal coordinate sequence of the moving object. Among them, the first - dimension features include joint range of motion, relative centroid position, spine trajectory, and angular velocity of joint range of motion. The first - dimension additional features include the relative position of the lateral elbow joint and the body's longitudinal line, the relative position of the lateral shoulder and elbow, and the relative position of the lateral elbow joint.

[0042] Judge whether the movement is symmetric through the spine trajectory, judge whether the movement is stable through the relative centroid position, judge whether the movement has contralateral coordination and ipsilateral coordination through the joint range of motion, judge whether the movement has maneuverability through the joint range of motion and angular velocity, and take symmetry, stability, contralateral coordination degree, ipsilateral coordination degree, and maneuverability as the second - dimension features.

[0043] Split each scoring item in the clinical scale into multiple first - dimension features and / or first - dimension additional features, score each scoring item according to the values of the first - dimension features and the values of the first - dimension additional features, and take the total score of all scoring items as the third - dimension feature. Among them, the clinical scale includes scoring items and the orientations of the video acquisition devices corresponding to the scoring items.

[0044] Set feature weights for each feature in the second - dimension features and the third - dimension feature respectively, and weight each feature in the second - dimension features and the third - dimension feature according to the feature weights to obtain the evaluation result of the motor function.

[0045] In specific implementation, in order to extract the basic features (as the first - dimension features) in the motor function, the following steps are used to obtain the first - dimension features at each orientation angle according to the skeletal coordinate sequence at each orientation angle:

[0046] Extract the coordinates of each joint point from the skeletal coordinate sequence at each orientation angle, connect the coordinates to form a set of spatial vectors, calculate the degree of the included angle formed by the spatial vectors, take the degree of the included angle as the joint range of motion of each joint, divide the skeleton diagram of the target object into different segments, calculate the centroid position of each segment, and then weight each segment centroid position with the relative mass distribution coefficient as the weight to obtain the relative centroid position of the target object; according to the skeletal coordinate sequence at each orientation angle, calculate the mid - points of the two shoulders and the mid - points of the two hips, take the spatial vector formed by connecting the coordinates of the mid - points of the two shoulders and the mid - points of the two hips as the spine vector, normalize the spine vector to generate a normalized spine vector, and generate a spine trajectory in the order of the frames of the image data; take the frame of the image data as the unit, calculate the difference in joint range of motion between adjacent frames as the angular velocity of each joint range of motion; according to the skeletal coordinate sequence at each orientation angle, calculate the relative position of the lateral elbow joint and the body's longitudinal line, the relative position of the lateral shoulder and elbow, and the relative position of the lateral elbow joint.

[0047] Specifically, the first-dimensional features include joint range of motion, joint range of motion, spinal trajectory, angular velocity of joint range of motion, relative position between the elbow joint and the body's longitudinal line, angle between the forearm and the horizontal line, and relative position between the thumb and the little finger.

[0048] Specifically, the joint range of motion includes the range of motion of the left shoulder, the range of motion of the right shoulder, the range of motion of the left elbow, the range of motion of the right elbow, the range of motion of the left hip, the range of motion of the right hip, the range of motion of the left knee, the range of motion of the right knee, the range of motion of the left wrist, and the range of motion of the right wrist.

[0049] Specifically, for the bone coordinate sequence (left shoulder p1, right shoulder p2, left elbow p3, right elbow p4, left wrist p5, right wrist p6, left hip p7, right hip p8, left knee p9, right knee p 10 , left ankle p 11 , right ankle p 12 , left ear p 13 , right ear p 14 , left eye p 15 , right eye p 16 , left middle finger metacarpophalangeal joint p 17 , right middle finger metacarpophalangeal joint p 18 , left thumb tip p 19 , right thumb tip p 20 , left little finger tip p 21 , right little finger tip p 22 ), the range of motion of the left shoulder, the range of motion of the right shoulder, the range of motion of the left elbow, the range of motion of the right elbow, the range of motion of the left hip, the range of motion of the right hip, the range of motion of the left knee, the range of motion of the right knee, the range of motion of the left wrist, and the range of motion of the right wrist are calculated respectively.

[0050] Specifically, from the coordinate points p7, p1, p3 of the left hip, left shoulder, and left elbow, the spatial vector is obtained, and the included angle θ between the vectors is calculated as θ = atan 2(y2 - y1, x2 - x1), which is used as the range of motion of the left shoulder joint.

[0051] Specifically, from the coordinate points p8, p2, p4 of the right hip, right shoulder, and right elbow, the spatial vector is obtained, and the included angle θ between the vectors is calculated as θ = atan 2(y2 - y1, x2 - x1), which is used as the range of motion of the right shoulder joint.

[0052] Specifically, from the coordinate points p1, p3, p5 of the left shoulder, left elbow, and left wrist, the spatial vector is obtained, and the included angle θ between the vectors is calculated as θ = atan 2(y2 - y1, x2 - x1), which is used as the range of motion of the left elbow joint.

[0053] Specifically, from the coordinate points p2, p4, p5 of the right shoulder, right elbow, and right wrist, the spatial vector Calculate the included angle θ between vectors as θ = atan 2(y2 - y1, x2 - x1), which serves as the range of motion of the right elbow joint.

[0054] Specifically, from the coordinate points p1, p7, p9 of the left shoulder, left hip, and left knee, obtain the spatial vector Calculate the included angle θ between vectors as θ = atan 2(y2 - y1, x2 - x1), which serves as the range of motion of the left hip joint.

[0055] Specifically, from the coordinate points p2, p8, p of the right shoulder, right hip, and right knee 10 , obtain the spatial vector Calculate the included angle θ between vectors as θ = atan 2(y2 - y1, x2 - x1), which serves as the range of motion of the right hip joint.

[0056] Specifically, from the coordinate points p7, p9, p of the left hip, left knee, and left wrist 11 , obtain the spatial vector Calculate the included angle θ between vectors as θ = atan 2(y2 - y1, x2 - x1), which serves as the range of motion of the left knee joint.

[0057] Specifically, from the coordinate points p8, p of the right hip, right knee, and right wrist 10 , p 12 , obtain the spatial vector Calculate the included angle θ between vectors as θ = atan 2(y2 - y1, x2 - x1), which serves as the range of motion of the right knee joint.

[0058] From the coordinate points p3, p5, p of the left elbow, left wrist, and the metacarpophalangeal joint of the left middle finger 17 , obtain the spatial vector Calculate the included angle θ between vectors as θ = atan 2(y2 - y1, x2 - x1), which serves as the range of motion of the left wrist joint.

[0059] Specifically, from the coordinate points p4, p6, p of the right elbow, right wrist, and the metacarpophalangeal joint of the right middle finger 18 , obtain the spatial vector Calculate the included angle θ between vectors as θ = atan 2(y2 - y1, x2 - x1), which serves as the range of motion of the right wrist joint.

[0060] Specifically, the piecewise centroid positions include the centroid positions of the left calf, right calf, left thigh, right thigh, left forearm, right forearm, left upper arm, right upper arm, and torso.

[0061] Specifically, each body segment of the human body is divided into upper and lower points. The upper and lower points are multiplied by their respective weights and added to obtain the relative centroid position of the body segment.

[0062] Specifically, take the left knee p9(x1, y1) and the left ankle p 11 (x2, y2), and calculate the centroid position of the left lower leg as (A * x1 + B * x2, A * y1 + B * y2) according to the relative weights A and B of the two.

[0063] Specifically, take the right knee P 10 (x1, y1) and the right ankle p 12 (x2, y2), and calculate the centroid position of the right lower leg as: (A * x1 + B * x2, A * y1 + B * y2) according to the relative weights A and B of the two.

[0064] Specifically, take the left hip p7(x1, y1) and the left knee p9(x2, y2), and calculate the centroid position of the left thigh as: (C * x1 + D * x2, C * y1 + D * y2) according to the relative weights C and D of the two.

[0065] Specifically, take the right hip p8(x1, y1) and the right knee p 10 (x2, y2), and calculate the centroid position of the right thigh as: (C * x1 + D * x2, C * y1 + D * y2) according to the relative weights C and D of the two.

[0066] Specifically, take the left elbow p3(x1, y1) and the left wrist p5(x2, y2), and calculate the centroid position of the left forearm as: (E * x1 + F * x2, E * y1 + F * y2) according to the relative weights E and F of the two.

[0067] Specifically, take the right elbow p4(x1, y1) and the right wrist p6(x2, y2), and calculate the centroid position of the right forearm as: (E * x1 + F * x2, E * y1 + F * y2) according to the relative weights E and F of the two.

[0068] Specifically, take the left shoulder p1(x1, y1) and the left elbow p3(x2, y2), and calculate the centroid position of the left upper arm as: (G * x1 + H * x2, G * y1 + H * y2) according to the relative weights G and H of the two.

[0069] Specifically, take the right shoulder p2(x1, y1) and the right elbow p4(x2, y2), and calculate the centroid position of the right upper arm as: (G * x1 + H * x2, G * y1 + H * y2) according to the relative weights G and H of the two.

[0070] Specifically, take the left shoulder p1(x1, y1), right shoulder p2(x2, y2), left hip p7(x3, y3), and right hip p8(x4, y4). According to the weights I and J, calculate the centroid position of the torso: (I * x1 + I * x2 + J * x3 + J * x4, J * y1 + J * y2 + J * y3 + J * y4). After calculating the centroid position of each segment, use the relative mass distribution coefficient as the weight to perform weighted calculation on the centroid position of each segment to obtain the relative centroid position of the patient.

[0071] Specifically, according to the midpoint A(x1, y1) of the two shoulders and the midpoint B(x2, y2) of the two hips, calculate the spinal vector Spinal vector The length of is Normalize the spinal vector Obtain the normalized spinal vector. Plot the normalized spinal vector in the rectangular coordinate system according to the frame order, and the spinal trajectory can be obtained immediately.

[0072] Specifically, for the range of motion of each joint, subtract frame by frame to obtain the angular velocity of the range of motion of the joint, with the unit of degrees / frame.

[0073] During specific implementation, in order to judge whether there is symmetry through the joint point data, the following steps are used to judge the second-dimensional feature of the target object according to the first-dimensional feature:

[0074] According to the spinal trajectory, calculate the maximum yaw angle α on the left side of the coordinate axis and the maximum yaw angle β on the right side of the spinal trajectory respectively; calculate the yaw symmetry degree according to the maximum yaw angle α and the maximum yaw angle β; when the yaw symmetry degree is less than or equal to the yaw threshold, judge that the movement of the target object is symmetric, otherwise, judge that the movement of the target object is not symmetric.

[0075] Specifically,

[0076] During specific implementation, in order to calculate the second-dimensional feature through the first-dimensional feature, the following steps are used to judge the second-dimensional feature of the target object according to the first-dimensional feature:

[0077] Taking the frames that affect the data as units, calculate the centroid position differences of the segmented centroid positions of each segment in adjacent frames respectively to obtain the centroid position differences of different segments; when the centroid position differences of each segment are respectively greater than the centroid thresholds of their respective segments, judge that the movement of the target object is not stable; when the centroid position differences of each segment are respectively less than or equal to the centroid thresholds of their respective segments, judge that the movement of the target object is stable.

[0078] Specifically, the center of mass motion trajectory is used as stability. During stable motion, the center of mass of the human body changes very slowly. When the center of mass changes dramatically, it means that the patient has unstable conditions such as falling. The center of mass difference within a unit frame is calculated. When it is greater than a certain threshold, it is judged that the patient (target object) has unstable motion, indicating a risk.

[0079] Specifically, the center of mass position difference includes the left calf center of mass difference, the right calf center of mass difference, the left thigh center of mass difference, the right thigh center of mass difference, the left forearm center of mass difference, the right forearm center of mass difference, the left upper arm center of mass difference, the right upper arm center of mass difference and the torso center of mass difference.

[0080] In specific implementation, in order to determine whether there is coordination (including ipsilateral and contralateral) through joint point data, the following steps are performed to determine the second dimensional features of the target object based on the first dimensional features:

[0081] The same type of joints on the opposite side of the target object are taken as a group of opposite side joints, and the frames of the impact data are taken as the units, and the joint activities of the two joints in a group of opposite side joints are taken as the horizontal axis and the vertical axis respectively, to generate the opposite side activity curve, and calculate the distance between the opposite side activity curve and the preset opposite side curve, and use it as the first distance; the first distance is less than or equal to the corresponding opposite side threshold as the first condition, when all the groups of opposite side joints meet the first condition, it is judged that the movement of the target object has the opposite side coordination, otherwise, it is judged that the movement of the target object does not have the opposite side coordination; the two adjacent joints on the same side of the target object are taken as a group of opposite side joints, and the frames of the impact data are taken as the units, and the joint activities of the two joints in a group of opposite side joints are taken as the horizontal axis and the vertical axis respectively, to generate the opposite side activity curve, and calculate the distance between the opposite side activity curve and the preset opposite side curve, and use it as the second distance; the second distance is less than or equal to the corresponding opposite side threshold as the second condition, when all the groups of opposite side joints meet the second condition, it is judged that the movement of the target object has the opposite side coordination, otherwise, it is judged that the movement of the target object does not have the opposite side coordination.

[0082] Specifically, a set of contralateral joints includes left knee-right knee, left hip-right hip, left elbow-right elbow, and left shoulder-right shoulder.

[0083] Specifically, calculate the Euclidean distance between the left knee range of motion - right knee range of motion curve of the patient and the curve of a normal person as the similarity. If it is greater than the threshold, the movement of the target object does not have contralateral coordination; calculate the Euclidean distance between the left hip range of motion - right hip range of motion curve of the patient and the curve of a normal person as the similarity. If it is greater than the threshold, the movement of the target object does not have contralateral coordination; calculate the Euclidean distance between the left elbow range of motion - right elbow range of motion curve of the patient and the curve of a normal person as the similarity. If it is greater than the threshold, the movement of the target object does not have contralateral coordination; calculate the Euclidean distance between the left shoulder range of motion - right shoulder range of motion curve of the patient and the curve of a normal person as the similarity. If it is greater than the threshold, the movement of the target object does not have contralateral coordination.

[0084] Specifically, a group of ipsilateral joints includes left hip - left knee, right hip - right knee, left shoulder - left elbow, and right shoulder - right elbow.

[0085] Specifically, calculate the Euclidean distance between the left hip range of motion - left knee range of motion curve of the patient and the curve of a normal person as the similarity. If it is greater than the threshold, the movement of the target object does not have contralateral coordination; calculate the Euclidean distance between the right hip range of motion - right knee range of motion curve of the patient and the curve of a normal person as the similarity. If it is greater than the threshold, the movement of the target object does not have contralateral coordination; calculate the Euclidean distance between the left shoulder range of motion - left elbow range of motion curve of the patient and the curve of a normal person as the similarity. If it is greater than the threshold, the movement of the target object does not have contralateral coordination; calculate the Euclidean distance between the right shoulder range of motion - right elbow range of motion curve of the patient and the curve of a normal person as the similarity. If it is greater than the threshold, the movement of the target object does not have contralateral coordination.

[0086] In specific implementation, in order to determine whether there is mobility based on joint point data, the following steps are used to determine the second - dimension feature of the target object according to the first - dimension feature:

[0087] Take the joint range of motion and the corresponding angular velocity of the same joint as a group of joint data. Taking the frame of the image data as the unit, use the joint range of motion and the corresponding angular velocity of each group of joint data as the horizontal axis and the vertical axis respectively to generate a mobility curve, calculate the distance between the mobility curve and the preset mobility curve, and use it as the third distance; use the third distance being less than or equal to the corresponding mobility threshold as the third condition. When the joint data of all groups meet the third condition, it is determined that the movement of the target object has mobility; otherwise, it is determined that the movement of the target object does not have mobility.

[0088] Specifically, the joints include the left shoulder, right shoulder, left elbow, right elbow, left hip, right hip, left knee, right knee, left wrist, and right wrist.

[0089] Specifically, joint range of motion - joint angular velocity of range of motion is used as mobility. Calculate the Euclidean distance between the curve of the patient's left shoulder range of motion - left shoulder joint angular velocity and the curve of a normal person as the similarity. If it is greater than the threshold, the movement of the target object does not have mobility; calculate the Euclidean distance between the curve of the patient's right shoulder range of motion - right shoulder joint angular velocity and the curve of a normal person as the similarity. If it is greater than the threshold, the movement of the target object does not have mobility; calculate the Euclidean distance between the curve of the patient's left elbow range of motion - left elbow joint angular velocity and the curve of a normal person as the similarity. If it is greater than the threshold, the movement of the target object does not have mobility; calculate the Euclidean distance between the curve of the patient's right elbow range of motion - right elbow joint angular velocity and the curve of a normal person as the similarity. If it is greater than the threshold, the movement of the target object does not have mobility; calculate the Euclidean distance between the curve of the patient's left hip range of motion - left hip joint angular velocity and the curve of a normal person as the similarity. If it is greater than the threshold, the movement of the target object does not have mobility; calculate the Euclidean distance between the curve of the patient's right hip range of motion - right hip joint angular velocity and the curve of a normal person as the similarity. If it is greater than the threshold, the movement of the target object does not have mobility; calculate the Euclidean distance between the curve of the patient's left knee range of motion - left knee joint angular velocity and the curve of a normal person as the similarity. If it is greater than the threshold, the movement of the target object does not have mobility; calculate the Euclidean distance between the curve of the patient's right knee range of motion - right knee joint angular velocity and the curve of a normal person as the similarity. If it is greater than the threshold, the movement of the target object does not have mobility; calculate the Euclidean distance between the curve of the patient's left wrist range of motion - left wrist joint angular velocity and the curve of a normal person as the similarity. If it is greater than the threshold, the movement of the target object does not have mobility; calculate the Euclidean distance between the curve of the patient's right wrist range of motion - right wrist joint angular velocity and the curve of a normal person as the similarity. If it is greater than the threshold, the movement of the target object does not have mobility.

[0090] Specifically, obtain the correspondence between the scoring items and the camera orientation according to Table 1 (Table 1 shows the viewing angle (camera orientation) and scoring criteria corresponding to each action).

[0091]

[0092]

[0093]

[0094]

[0095]

[0096] Table 1

[0097] Correspond the scoring items in Table 1 (the corresponding relationship between the scoring items and the camera orientations) with the first - dimension features and / or the first - dimension additional features (including the first - dimension features and / or the first - dimension additional features and the corresponding camera orientations) one by one to generate Table 2.

[0098]

[0099]

[0100]

[0101] Table 2

[0102] According to the corresponding relationship in Table 2, obtain the data of the first - dimension features (such as the range of motion of the wrist joint, etc.) in the front view and / or the side view respectively. Set the corresponding numerical thresholds for the scores (0, 1, 2 points) in Table 1. Determine whether the data of the first - dimension features is within the numerical threshold range corresponding to the scores (0, 1, 2 points), so as to obtain the scores of each scoring item. Sum up the scores of all scoring items, and take the total score as the third - dimension feature.

[0103] Specifically, score each scoring item according to the value of the first - dimension features and / or the value of the first - dimension additional features, including: setting numerical intervals for the value of the first - dimension features and / or the value of the first - dimension additional features respectively; determining whether the value is within the numerical interval, and scoring each scoring item according to the judgment result.

[0104] In specific implementation, in order to obtain the evaluation result by separately weighting the second - dimension feature and the third - dimension feature, the following steps are used to determine the evaluation result of the motion function of the target object according to the second - dimension feature and the third - dimension feature:

[0105] Set feature weights for each feature in the second - dimension feature and the third - dimension feature respectively, and weight each feature in the second - dimension feature and the third - dimension feature according to the feature weights to determine the evaluation result of the motion function of the target object.

[0106] Specifically, set the symmetry of the second - dimension feature with a weight of 0.1, the stability with a weight of 0.1, the contralateral coordination with a weight of 0.1, the ipsilateral coordination with a weight of 0.1, the mobility with a weight of 0.1, and the third - dimension feature with a weight of 0.5. In a certain motion of the target object, if it has symmetry, does not have stability, has contralateral coordination, does not have ipsilateral coordination, has mobility, and the total score of the third - dimension feature is 60 points, then the evaluation result of the motion function of the target object is (10 + 0+10 + 0+10 + 60*0.5).

[0107] Specifically, the criteria for the comprehensive evaluation of motor ability are based on big data. Using the deviation method according to the evaluation results, five levels of evaluation are given: excellent, good, medium, slightly poor, and poor. For accurate evaluation, the main problems of the target object are summarized according to each feature and presented in the report.

[0108] In this embodiment, a computer device is provided. As Figure 6 shown, it includes a memory 601, a processor 602, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements any of the above-mentioned multi-dimensional motor function evaluation methods.

[0109] Specifically, the computer device can be a computer terminal, a server, or a similar computing device.

[0110] In this embodiment, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program for executing any of the above-mentioned multi-dimensional motor function evaluation methods.

[0111] Specifically, the computer-readable storage medium includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer-readable storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD), or other optical storage, magnetic cassette tapes, magnetic disk storage, or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device. As defined herein, computer-readable storage media do not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0112] Based on the same inventive concept, an evaluation device for multi-dimensional motor functions is also provided in the embodiments of the present invention, as described in the following embodiments. Since the principle of solving problems by the evaluation device for multi-dimensional motor functions is similar to that of the evaluation method for multi-dimensional motor functions, the implementation of the evaluation device for multi-dimensional motor functions can refer to the implementation of the evaluation method for multi-dimensional motor functions, and the repeated parts will not be described again. As used hereinafter, the term "unit" or "module" can be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0113] Figure 7 is a structural block diagram of an evaluation device for multi-dimensional motion functions according to an embodiment of the present invention. As Figure 7 shown, it includes: a data acquisition module 701, a module 702 for obtaining a sequence of bone coordinates, a first-dimensional feature calculation module 703, a second-dimensional feature module 704, a third-dimensional feature calculation module 705, and an evaluation report generation module 706. The following is an explanation of this structure.

[0114] The data acquisition module 701 is configured to respectively acquire image data of a target object at multiple azimuth angles during the movement of the target object;

[0115] The module 702 for obtaining a sequence of bone coordinates is configured to obtain a sequence of bone coordinates of the target object at each azimuth angle according to the image data;

[0116] The first-dimensional feature calculation module 703 is configured to obtain first-dimensional features at each azimuth angle according to the sequence of bone coordinates at each azimuth angle, where the first-dimensional features include multiple joint movement-related features and relative position features;

[0117] The second-dimensional feature judgment module 704 is configured to judge the second-dimensional features of the target object according to the first-dimensional features, where the second-dimensional features include multiple evaluation index features related to movement ability;

[0118] The third-dimensional feature calculation module 705 is configured to calculate the sum of the scores corresponding to each first-dimensional feature according to the correspondence between the first-dimensional features and the scores, and use the sum as the third-dimensional feature;

[0119] The evaluation report generation module 706 is configured to determine the evaluation result of the movement function of the target object according to the second-dimensional features and the third-dimensional features.

[0120] In one embodiment, the first-dimensional feature calculation module includes:

[0121] The joint range of motion calculation unit is configured to extract the coordinates of each joint point from the sequence of bone coordinates at each azimuth angle, form a set of spatial vectors by connecting the coordinates, calculate the degree of the included angle formed by the spatial vectors, and use the degree of the included angle as the joint range of motion of each joint;

[0122] The centroid relative position calculation unit is configured to divide the skeleton diagram of the target object into different segments, calculate the centroid position of each segment, and then weight each segment centroid position with the relative mass distribution coefficient as the weight to obtain the centroid relative position of the target object;

[0123] A spine trajectory generation unit, which is used to calculate the midpoints of the two shoulders and the midpoints of the two hips according to the bone coordinate sequences at various azimuth angles, use the spatial vector formed by connecting the coordinates of the midpoints of the two shoulders and the midpoints of the two hips as the spine vector, normalize the spine vector to generate a normalized spine vector, and generate a spine trajectory according to the order of the frames of the image data;

[0124] An angular velocity calculation unit, which is used to calculate the difference in joint mobility between adjacent frames as the angular velocity of each joint mobility in units of the frames of the image data;

[0125] A first-dimension additional feature calculation unit, which is used to calculate the relative position of the lateral elbow joint and the body longitudinal line, the relative position of the lateral shoulder and the elbow, and the relative position of the lateral elbow joint according to the bone coordinate sequences at various azimuth angles.

[0126] In one embodiment, the second-dimension feature judgment module includes:

[0127] A maximum yaw angle calculation unit, which is used to calculate the maximum yaw angle α on the left side of the coordinate axis and the maximum yaw angle β on the right side of the coordinate axis of the spine trajectory respectively according to the spine trajectory;

[0128] A yaw symmetry calculation unit, which is used to calculate the yaw symmetry according to the maximum yaw angle α and the maximum yaw angle β;

[0129] A symmetry judgment unit, which is used to judge that the movement of the target object is symmetric when the yaw symmetry is less than or equal to the yaw threshold, otherwise, judge that the movement of the target object is not symmetric.

[0130] In one embodiment, the second-dimension feature judgment module further includes:

[0131] A centroid position difference calculation unit, which is used to calculate the centroid position differences of the sectional centroids of each segment between adjacent frames respectively in units of the frames of the influence data to obtain the centroid position differences of different segments;

[0132] An instability judgment unit, which is used to judge that the movement of the target object is not stable when the centroid position differences of each segment are respectively greater than the centroid thresholds of their respective segments;

[0133] A stability judgment unit, which is used to judge that the movement of the target object is stable when the centroid position differences of each segment are respectively less than or equal to the centroid thresholds of their respective segments.

[0134] In one embodiment, the second-dimension feature judgment module further includes:

[0135] The first distance calculation unit is configured to use the same - type joints on the opposite side of the target object as a group of opposite - side joints. Taking the frame of the influencing data as a unit, taking the joint mobility of the two joints on both sides in a group of opposite - side joints as the horizontal axis and the vertical axis respectively, generating an opposite - side mobility curve, calculating the distance between the opposite - side mobility curve and a preset opposite - side curve, and taking it as the first distance;

[0136] The opposite - side coordination judgment unit is configured to take that the first distance is less than or equal to the corresponding opposite - side threshold as the first condition. When all groups of opposite - side joints meet the first condition, it is judged that the movement of the target object has opposite - side coordination; otherwise, it is judged that the movement of the target object does not have opposite - side coordination;

[0137] The second distance calculation unit is configured to use two adjacent joints on the same side of the target object as a group of same - side joints. Taking the frame of the influencing data as a unit, taking the joint mobility of the two joints in a group of same - side joints as the horizontal axis and the vertical axis respectively, generating a same - side mobility curve, calculating the distance between the same - side mobility curve and a preset same - side curve, and taking it as the second distance;

[0138] The same - side coordination judgment unit is configured to take that the second distance is less than or equal to the corresponding same - side threshold as the second condition. When all groups of same - side joints meet the second condition, it is judged that the movement of the target object has same - side coordination; otherwise, it is judged that the movement of the target object does not have same - side coordination.

[0139] In one embodiment, the second - dimension feature judgment module further includes:

[0140] The third distance calculation unit is configured to use the joint mobility and the corresponding angular velocity of the same joint as a group of joint data. Taking the frame of the image data as a unit, taking the joint mobility and the corresponding angular velocity of each group of joint data as the horizontal axis and the vertical axis respectively, generating a mobility curve, calculating the distance between the mobility curve and a preset mobility curve, and taking it as the third distance;

[0141] The mobility judgment unit is configured to take that the third distance is less than or equal to the corresponding mobility threshold as the third condition. When all groups of joint data meet the third condition, it is judged that the movement of the target object has mobility; otherwise, it is judged that the movement of the target object does not have mobility.

[0142] In one embodiment, the evaluation report generation module further includes:

[0143] The evaluation result determination unit is configured to set feature weights for each feature in the second - dimension feature and the third - dimension feature respectively, weight each feature in the second - dimension feature and the third - dimension feature according to the feature weights, and determine the evaluation result of the movement function of the target object.

[0144] The embodiments of the present invention achieve the following technical effects:

[0145] Collect the image data of the target object from multiple angles, select videos from different angles when calculating different features, so that the motion information of the patient can be compared from multiple angles, making the evaluation result more accurate; evaluate the motor function through multiple dimensions of the first dimension feature, the second dimension feature and the third dimension feature, and the obtained evaluation result is three-dimensional and highly accurate; the second dimension feature adopts evaluation index features related to multiple motor abilities, and symmetry, stability, coordination and mobility can effectively evaluate the motor function from various angles; different from the prior art when using the FMA scoring standard for scoring, the scoring requires subjective judgment of people. In this embodiment, the evaluation method compares the data calculated by the first dimension feature with the scoring interval, and automatically calculates the score (as the third dimension feature), and the obtained score is more objective and accurate.

[0146] Obviously, those skilled in the art should understand that the above-mentioned modules or steps of the embodiments of the present invention can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. Optionally, they can be implemented by program codes executable by the computing device. Thus, they can be stored in the storage device and executed by the computing device. And in some cases, the steps shown or described can be executed in a different order than here, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module to implement. In this way, the embodiments of the present invention are not limited to any specific combination of hardware and software.

[0147] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, various changes and modifications can be made to the embodiments of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. An evaluation method for multi-dimensional motion functions, characterized in that Including: During the movement of the target object, image data of the target object is collected at multiple azimuth angles respectively; According to the image data, a sequence of bone coordinates of the target object at each azimuth angle is obtained; According to the sequences of bone coordinates at each azimuth angle, first-dimensional features at each azimuth angle are obtained, where the first-dimensional features include multiple joint movement-related features and relative position features; Judging the second-dimensional features of the target object according to the first-dimensional features, where the second-dimensional features include multiple evaluation index features related to motor ability; According to the corresponding relationship between the first-dimensional features and scores, calculate the sum of the scores corresponding to each of the first-dimensional features, and use the sum as the third-dimensional feature; Determine the evaluation result of the movement function of the target object according to the second-dimensional feature and the third-dimensional feature.

2. The evaluation method for multi-dimensional motion functions according to claim 1, wherein Obtaining the first-dimensional features at each azimuth angle according to the sequences of bone coordinates at each azimuth angle includes: Extract the coordinates of each joint point from the sequences of bone coordinates at each azimuth angle, connect the coordinates to form a set of spatial vectors, calculate the degree of the angle formed by the spatial vectors, and use the degree of the angle as the joint range of motion of each joint; Divide the skeleton diagram of the target object into different segments, calculate the centroid position of each segment, and then weight each centroid position of the segment with the relative mass distribution coefficient as the weight to obtain the relative centroid position of the target object; According to the sequences of bone coordinates at each azimuth angle, calculate the midpoints of the two shoulders and the midpoints of the two hips, use the spatial vector formed by connecting the coordinates of the midpoints of the two shoulders and the midpoints of the two hips as the spine vector, normalize the spine vector to generate a normalized spine vector, and generate a spine trajectory according to the order of the frames of the image data; Taking the frames of the image data as units, calculate the difference in the joint range of motion between adjacent frames as the angular velocity of each joint range of motion; According to the sequences of bone coordinates at each azimuth angle, calculate the relative position of the lateral elbow joint and the body longitudinal line, the relative position of the lateral shoulder and elbow, and the relative position of the lateral elbow joint.

3. The evaluation method for multi-dimensional motion functions according to claim 2, characterized in that, Judging the second-dimensional features of the target object according to the first-dimensional features includes: According to the spine trajectory, calculate the maximum yaw angle α on the left side of the coordinate axis and the maximum yaw angle β on the right side of the coordinate axis of the spine trajectory respectively; Calculate the yaw symmetry degree according to the maximum yaw angle α and the maximum yaw angle β; When the yaw symmetry degree is less than or equal to the yaw threshold, it is judged that the movement of the target object is symmetric, otherwise, it is judged that the movement of the target object is not symmetric.

4. The evaluation method for multi-dimensional motion functions according to claim 2, characterized in that, Judging the second-dimensional features of the target object according to the first-dimensional features includes: Taking the frames of the influence data as units, calculate the difference in the centroid position of each segment between adjacent frames respectively to obtain the difference in the centroid position of different segments; When the difference in the centroid position of each segment is greater than the centroid threshold of its respective segment, it is judged that the movement of the target object is not stable; When the difference in the centroid positions of each segment is less than or equal to the centroid threshold of its respective segment, it is determined that the movement of the target object is stable.

5. The evaluation method for multi-dimensional motion functions according to claim 2, wherein Judging the second dimension feature of the target object according to the first dimension feature includes: Taking the same type of joints on the opposite side of the target object as a group of opposite-side joints. Taking the frame of the influence data as the unit, taking the joint mobility of the two joints on both sides in a group of opposite-side joints as the horizontal axis and the vertical axis respectively, generating an opposite-side mobility curve, calculating the distance between the opposite-side mobility curve and the preset opposite-side curve, and taking it as the first distance; Taking the first distance being less than or equal to the corresponding opposite-side threshold as the first condition. When all groups of opposite-side joints meet the first condition, it is determined that the movement of the target object has opposite-side coordination. Otherwise, it is determined that the movement of the target object does not have opposite-side coordination; Taking two adjacent joints on the same side of the target object as a group of same-side joints. Taking the frame of the influence data as the unit, taking the joint mobility of the two joints in a group of same-side joints as the horizontal axis and the vertical axis respectively, generating a same-side mobility curve, calculating the distance between the same-side mobility curve and the preset same-side curve, and taking it as the second distance; Taking the second distance being less than or equal to the corresponding same-side threshold as the second condition. When all groups of same-side joints meet the second condition, it is determined that the movement of the target object has same-side coordination. Otherwise, it is determined that the movement of the target object does not have same-side coordination.

6. The evaluation method for multi-dimensional motion function according to claim 2, characterized in that Judging the second dimension feature of the target object according to the first dimension feature includes: Taking the joint mobility and the corresponding angular velocity of the same joint as a group of joint data. Taking the frame of the image data as the unit, taking the joint mobility and the corresponding angular velocity of each group of joint data as the horizontal axis and the vertical axis respectively, generating a maneuverability curve, calculating the distance between the maneuverability curve and the preset maneuverability curve, and taking it as the third distance; Taking the third distance being less than or equal to the corresponding maneuverability threshold as the third condition. When all groups of joint data meet the third condition, it is determined that the movement of the target object has maneuverability. Otherwise, it is determined that the movement of the target object does not have maneuverability.

7. The evaluation method for multi-dimensional motion functions according to any one of claims 1 to 6, characterized in that, Determining the evaluation result of the movement function of the target object according to the second dimension feature and the third dimension feature includes: Respectively setting feature weights for each feature in the second dimension feature and the third dimension feature, and weighting each feature in the second dimension feature and the third dimension feature according to the feature weights to determine the evaluation result of the movement function of the target object.

8. An evaluation device for multi-dimensional motion functions, characterized in that, Including: A data acquisition module, configured to respectively acquire image data of the target object at multiple azimuth angles during the movement of the target object; An obtaining bone coordinate sequence module, configured to obtain the bone coordinate sequence of the target object at each azimuth angle according to the image data; A first dimension feature calculation module, configured to obtain the first dimension feature at each azimuth angle according to the bone coordinate sequence at each azimuth angle, where the first dimension feature includes multiple joint movement-related features and relative position features; The second-dimensional feature judgment module is used to judge the second-dimensional features of the target object according to the first-dimensional features, wherein the second-dimensional features include multiple evaluation index features related to motor ability; The third-dimensional feature calculation module is used to calculate the sum of the scores corresponding to each of the first-dimensional features according to the correspondence between the first-dimensional features and the scores, and use the sum as the third-dimensional feature; The evaluation report generation module is used to determine the evaluation result of the motor function of the target object according to the second-dimensional features and the third-dimensional features.

9. A computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the multi-dimensional motor function evaluation method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program for executing the multi-dimensional motor function evaluation method according to any one of claims 1 to 7.

Images