Limb dyskinesia recognition and active movement auxiliary control method based on computer vision

Abstract

The invention provides a limb dyskinesia recognition and active exercise auxiliary control method based on computer vision, and relates to the technical field of exercise rehabilitation engineering. According to the method, the motion image sequence of the upper limbs and / or the lower limbs of the patient is obtained, the multi-joint three-dimensional coordinates are extracted to form the spatial position data, and the joint angles and the angular speeds are calculated to obtain the joint motion state data. Combining with the torque support evidence, identifying joint angle deviation, angular velocity abnormity and compensation behaviors, generating a dyskinesia identification result containing intention intensity and compensation indexes, and determining target auxiliary joint and auxiliary torque output according to the dyskinesia identification result; and active motion auxiliary control is implemented in modes of joint driving assistance, torque compensation, track amplification or lateral force compensation and the like. According to the invention, fine recognition and self-adaptive assistance of dyskinesia can be realized without additional wearable equipment, and the accuracy and safety of rehabilitation training are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of sports rehabilitation engineering, in particular to a limb movement disorder recognition and active movement auxiliary control method based on computer vision. BACKGROUND

[0002] With the increase of the aging society and the incidence of diseases such as stroke and trauma, the number of patients with upper and lower limb movement disorders continues to rise. Traditional rehabilitation training methods mainly rely on manual stretching and passive activities by rehabilitation therapists, and the training intensity and rhythm are greatly influenced by human experience, and the training process is difficult to quantify and record for a long time. In recent years, intelligent rehabilitation devices such as rehabilitation robots and exoskeletons have emerged, which can achieve high repeatability and high precision training control to a certain extent, but most of the current systems mainly rely on joint encoders, position / sensor or electromyographic signals to determine the movement state of the patient, which is difficult to comprehensively perceive the posture changes of multiple joints such as shoulders, elbows, wrists, hips, knees and ankles in the spatial dimension, especially to distinguish the real movement ability and compensatory movement of the patient.

[0003] With the development of computer vision and kinematics modeling technology, using visual sensing technology to recognize and analyze human multi-joint three-dimensional movement has become an important development direction in the field of rehabilitation. In the rehabilitation training scene, how to obtain the spatial coordinates and joint angle information of the key joints of the patient's upper and lower limbs in real time without increasing the additional mechanical burden, combined with the expected movement trajectory and real-time movement performance, to identify movement disorders and compensatory behaviors, and to adaptively adjust the output of the rehabilitation robot, is an important technical path to improve the safety and effectiveness of training. Therefore, the integrated method combining visual perception, movement disorder recognition and active movement auxiliary control has become an important development trend of rehabilitation robot and exoskeleton control system.

[0004] The existing rehabilitation system relying on mechanical sensors or electromyographic signals is difficult to identify abnormal behaviors of the upper limbs such as shoulder shrugging, forward leaning or lateral bending of the trunk, limited or abnormal bending of the elbow joint, abnormal dorsiflexion or deviation of the wrist joint, and is also difficult to identify gait abnormalities of the lower limbs such as excessive swing or lifting of the hip joint, insufficient flexion or hyperextension of the knee joint, and foot drop or inversion of the ankle joint, so it is difficult to accurately determine whether the patient is in a movement disorder and whether there is obvious compensatory behavior, which may cause improper assistance and even movement injury. SUMMARY

[0005] In order to overcome the deficiencies of the prior art, the purpose of the present application is to provide a computer vision-based limb movement disorder recognition and active movement assistance control method, which acquires multi-joint three-dimensional movement information through computer vision and combines force and torque support evidence to achieve fine recognition of movement disorders and compensatory behaviors, outputs intention intensity and compensation index to drive adaptive and multi-strategy active movement assistance control, thereby significantly improving the accuracy, safety and effectiveness of rehabilitation training without increasing additional wearing burden.

[0006] In order to achieve the above-mentioned purpose, the present application provides the following scheme: A computer vision-based limb movement disorder recognition and active movement assistance control method, comprising: acquiring a movement image sequence of a patient's upper limbs and / or lower limbs, and extracting three-dimensional coordinates of trunk points, shoulder joints, elbow joints, wrist joints, and hip joints, knee joints, ankle joints and toe points to form multi-joint spatial position data; constructing limb multi-segment vectors and calculating vector angles according to the spatial position data to obtain joint angles and joint angular velocities, forming joint movement state data; judging joint angle deviation, angular velocity anomaly and torque anomaly according to the joint movement state data and combining force and torque support evidence, recognizing corresponding movement disorder types, determining whether there is trunk compensation or joint compensation behavior, outputting intention intensity and compensation index, and forming movement disorder recognition results; determining the direction and size of the target auxiliary joint and the auxiliary torque according to the movement disorder recognition results, generating movement assistance control instructions, and implementing active movement assistance control through at least one of joint driving assistance, torque compensation, trajectory amplification, mechanical limiting or damping, and lateral force compensation to guide the patient to complete the preset rehabilitation training action.

[0007] Preferably, constructing limb multi-segment vectors and calculating vector angles according to the spatial position data comprises: constructing an upper arm vector from the shoulder joint to the elbow joint, a forearm vector from the elbow joint to the wrist joint, and a trunk vector from the trunk point to the shoulder joint using the three-dimensional coordinates of the shoulder joint, elbow joint, wrist joint and trunk point in the spatial position data; calculating the included angle between the upper arm vector and the trunk vector and the included angle between the upper arm vector and the forearm vector to obtain the shoulder joint angle and the elbow joint angle; obtaining the shoulder joint angular velocity and the elbow joint angular velocity according to the changes of the shoulder joint angle and the elbow joint angle with time, so that the joint movement state data at least includes the shoulder joint angle, the shoulder joint angular velocity, the elbow joint angle and the elbow joint angular velocity.

[0008] Preferably, constructing limb multi-segment vectors according to the spatial position data and calculating the vector angles further comprises: constructing a thigh vector from the hip joint to the knee joint, a shank vector from the knee joint to the ankle joint, and a foot segment vector from the ankle joint to the toe point, using the three-dimensional coordinates of the hip joint, the knee joint, the ankle joint, and the toe point in the spatial position data; calculating the angles between the thigh vector and the shank vector, and the angles between the shank vector and the foot segment vector, to obtain the knee joint angle and the ankle joint angle, and obtaining the knee joint angular velocity and the ankle joint angular velocity according to the changes of the knee joint angle and the ankle joint angle over time; determining the hip joint angle and the hip joint angular velocity according to the position relationship of the hip joint in the spatial position data, so that the joint motion state data at least includes the hip joint angle, the hip joint angular velocity, the knee joint angle, the knee joint angular velocity, the ankle joint angle, and the ankle joint angular velocity.

[0009] Preferably, according to the joint motion state data and in combination with the force and torque support evidence, judging the joint angle deviation, the angular velocity anomaly, and the torque anomaly, and identifying the corresponding movement disorder type, comprises: in the joint motion state data, when the shoulder joint angle and the shoulder joint angular velocity are inconsistent with the preset shoulder joint motion range and change pattern, and the torso is determined to be out of the preset range in the vertical direction or the front-back direction according to the displacement of the torso point in the spatial position data, it is determined that there is a shoulder raising or a torso forward bending, lateral bending, or other torso compensation behavior; in the joint motion state data, when the elbow joint angle does not reach the expected range within a preset time, or the elbow joint angle change trend is opposite to the preset target direction, and the elbow joint angular velocity appears obvious decline or mutation, in combination with the force and torque support evidence, it is determined that there is a movement disorder of elbow joint flexion and extension limitation or abnormal bending; in the joint motion state data, when the wrist joint angle relative to the forearm continuously exceeds the preset normal range, or the wrist movement trajectory deviates from the target direction by more than the preset threshold according to the spatial position data, it is determined that there is a movement disorder of abnormal wrist joint dorsiflexion, palmar flexion, or deviation.

[0010] Preferably, according to the joint motion state data and in combination with the force and torque support evidence, judging the joint angle deviation, the angular velocity anomaly, and the torque anomaly, and identifying the corresponding movement disorder type, further comprises: in the swing phase of gait, when the maximum flexion value of the knee joint angle in the joint motion state data is lower than the preset flexion threshold, it is determined that there is a knee joint flexion and extension deficiency; When the knee angle in the joint motion state data exceeds the preset upper limit of straightening and shows a backward bending trend, and the knee-related force and moment support evidence shows abnormal stress, it is determined that there is a reverse knee; When the knee angular velocity in the joint motion state data frequently stalls, mutates or irregularly fluctuates in a high frequency within a continuous gait cycle, it is determined that there is a motion disorder of knee joint jamming.

[0011] Preferably, according to the joint motion state data, and in combination with the force and moment support evidence, judging the joint angle deviation, angular velocity anomaly and torque anomaly, identifying the corresponding motion disorder type also includes: When the ankle angle in the joint motion state data does not reach the preset dorsiflexion angle, and according to the spatial position data, it is determined that the foot segment is in a toe droop posture, it is determined that there is foot drop during the swing phase of gait; When the ankle angle in the joint motion state data continuously exceeds the preset range in the plantar flexion direction during standing or gait propulsion, and the ankle-related force and moment support evidence indicates weight-bearing instability, it is determined that there is excessive ankle plantar flexion; When the foot segment deviates from the normal direction of the ground in the varus or valgus direction and exceeds the preset range according to the spatial position data, it is determined that there is a motion disorder of ankle varus or valgus.

[0012] Preferably, according to the motion disorder identification result, the direction and size of the target auxiliary joint and auxiliary torque are determined, the motion assistance control instruction is generated, and at least one of the following modes is implemented to implement active motion assistance control: joint driving assistance, torque compensation, trajectory amplification, mechanical limiting or damping, and lateral force compensation, to guide the patient to complete the preset rehabilitation training action, including: When the motion disorder identification result indicates that there is a shoulder or trunk compensation behavior, the assistance force in the trunk lifting or forward leaning direction is reduced, and the assistance is adjusted to target the elbow joint as the target auxiliary joint and apply an auxiliary torque in the elbow flexion direction; When the motion disorder identification result indicates that there is elbow flexion limitation or abnormal bending, the elbow joint is set as the target auxiliary joint, the direction and size of the auxiliary torque are determined according to the deviation of the elbow joint angle and the elbow joint angular velocity from the preset target, and the elbow joint motion is made to approach the preset target trajectory; When the motion disorder identification result indicates that there is abnormal dorsiflexion or palmar flexion of the wrist joint, the wrist joint is set as the target auxiliary joint, and an auxiliary torque is applied at the end execution position to offset the abnormal dorsiflexion or palmar flexion, so that the wrist joint remains close to the neutral position.

[0013] Preferably, according to the movement disorder identification result, the target assisted joint and the direction and size of the assistive torque are determined, the movement assist control instruction is generated, and the active movement assist control is implemented by at least one of the following modes: joint driving assistance, torque compensation, trajectory amplification, mechanical limiting or damping, and lateral force compensation, to guide the patient to complete the preset rehabilitation training action, and the method further comprises: When the movement disorder identification result indicates that there is insufficient knee flexion and extension, the knee joint is set as the target assisted joint, an assistive torque is applied in the knee flexion direction during the swing phase of gait, and the knee flexion angle is amplified within a preset range; When the movement disorder identification result indicates that there is a reverse knee, the knee joint is set as the target assisted joint, a limiting assistive torque is applied in the knee flexion direction during the support phase of gait to inhibit excessive straightening and backward bending of the knee joint; When the movement disorder identification result indicates that there is knee joint jamming, a rhythmic and small-amplitude assistive torque is applied to the knee joint within a continuous gait cycle to improve the continuity and smoothness of the knee joint angle and the knee joint angular velocity.

[0014] Preferably, according to the movement disorder identification result, the target assisted joint and the direction and size of the assistive torque are determined, the movement assist control instruction is generated, and the active movement assist control is implemented by at least one of the following modes: joint driving assistance, torque compensation, trajectory amplification, mechanical limiting or damping, and lateral force compensation, to guide the patient to complete the preset rehabilitation training action, and the method further comprises: When the movement disorder identification result indicates that there is foot drop, the ankle joint is set as the target assisted joint, an assistive torque is applied in the ankle dorsiflexion direction during the swing phase of gait to lift the foot segment and reduce the risk of dragging the ground; When the movement disorder identification result indicates that there is excessive plantar flexion of the ankle joint, the ankle joint is set as the target assisted joint, an upper limit of the assistive torque or an equivalent damping is set in the plantar flexion direction, and support is provided in the ankle dorsiflexion direction when weight bearing is required; When the movement disorder identification result indicates that there is ankle inversion or eversion, the ankle joint is set as the target assisted joint, a lateral assistive force or a limiting assistive torque is applied in the inversion or eversion direction to maintain the balance of the foot bottom contact.

[0015] Preferably, the method further comprises: During multiple rehabilitation training processes, the intention intensity, the compensation index in the movement disorder identification result obtained each time, and the corresponding movement assist control instruction are recorded, and the selection of the target assisted joint and the direction and size of the assistive torque are adaptively adjusted according to the recorded content in subsequent training to optimize the effect of active movement assist control.

[0016] The application discloses the following technical effects: The application realizes three-dimensional, continuous and quantitative representation of the limb movement process by acquiring a movement image sequence of the upper limbs and / or lower limbs of a patient, extracting multi-joint three-dimensional coordinates and forming multi-joint spatial position data, and then obtaining joint angles and joint angular velocities through vector angle calculation to form joint movement state data. Compared with the prior art which mainly relies on empirical observation or single sensor signals, the application can cover trunk points, shoulder joints, elbow joints, wrist joints, hip joints, knee joints, ankle joints and toe points without increasing additional wearing burden, significantly improves the spatial resolution capability for complex upper limb movements and gait chains, provides a complete and reliable data basis for subsequent movement disorder identification, and overcomes the problems of "incomplete observation, unclear measurement and forgotten record" in the prior art.

[0017] Based on the joint movement state data, the application introduces force and torque support evidence to jointly judge joint angle deviation, angular velocity anomaly and torque anomaly, and further identifies trunk compensation or joint compensation behavior, so that different situations of "insufficient real ability of the patient" and "task completion through compensation actions such as shoulder raising, forward leaning and hip swinging" can be distinguished. Compared with the prior art which only judges the movement state according to joint encoders, end displacement or single electromyographic signals, the multi-dimensional evidence chain constructed by the application can earlier and more accurately find abnormal patterns such as shoulder raising, trunk forward leaning, knee reverse stretching, foot sagging, inversion and eversion, and effectively makes up for the defects of insufficient compensation behavior identification and rough movement disorder classification of the traditional system.

[0018] The application outputs intention strength and compensation index according to the joint movement state data and the force and torque support evidence, forms movement disorder identification results, and drives subsequent auxiliary decision-making, and establishes a closed loop link from "passive monitoring" to "quantitative evaluation to adaptive assistance". Compared with the prior art in which rehabilitation equipment can only output assistive force according to fixed programs or simple threshold rules, the application reflects the active intention of the patient through the intention strength, and represents the compensation degree through the compensation index, so that the assistive force, intervention time and intervention joint can be dynamically adjusted according to the state of the patient, thereby realizing individualized and phased optimization of the rehabilitation program, and avoiding the two-pole problems of "too much assistance" inhibiting active participation or "too little assistance" leading to training failure.

[0019] After obtaining the dyskinesia recognition result, the application generates a movement assistance control instruction containing a target auxiliary joint and an auxiliary torque direction and size for different joints and different dyskinesia types, and implements active movement assistance control in at least one of the following ways: joint driving assistance, torque compensation, trajectory amplification, mechanical limiting or damping, and lateral force compensation. Compared with the existing scheme that only provides single force position assistance or can only simply limit the joint activity range, the application can perform differentiated and refined control strategies on key joints such as shoulders, elbows, wrists, hips, knees, and ankles, which can not only enhance insufficient actions such as flexion and dorsiflexion, but also inhibit dangerous postures such as counter-zhang knee, excessive plantar flexion, abnormal inversion, etc., thereby improving the training effect while reducing the risk of secondary injury, and significantly improving the safety and effectiveness of rehabilitation training. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0021] Figure 1 The method flowchart provided for the embodiments of the present application. DETAILED DESCRIPTION

[0022] The technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0023] The purpose of the present application is to provide a kind of, by constructing "three-dimensional joint movement perception-disorder and compensation identification-precise auxiliary control" closed loop, accurate judgment and targeted assistance to patient real movement ability are realized, thereby significantly improve the intelligentization, adaptive and safety effect of rehabilitation training.

[0024] In order to make the above-mentioned purposes, characteristics and advantages of the present application more apparent and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0025] Figure 1 The method flowchart provided for the embodiments of the present application is shown in Figure 1, and the present application provides a computer vision-based limb dyskinesia recognition and active movement assistance control method, comprising: Figure 1 ​Step 100: Obtain a motion image sequence of the upper and / or lower limbs of the patient, and extract the three-dimensional coordinates of the trunk points, shoulder joints, elbow joints, wrist joints, and hip joints, knee joints, ankle joints and toe points to form multi-joint spatial position data; Step 200: Construct limb multi-segment vectors according to the spatial position data and calculate the vector angle to obtain joint angle and joint angular velocity, and form joint motion state data; Step 300: According to the joint motion state data, and combined with the force and torque support evidence, judge the joint angle deviation, angular velocity anomaly and torque anomaly, identify the corresponding movement disorder type, and determine whether there is trunk compensation or joint compensation behavior, output the intention strength and compensation index, and form the movement disorder recognition result; Step 400: According to the movement disorder recognition result, determine the direction and size of the target auxiliary joint and auxiliary torque, generate motion assistance control instructions, and implement active motion assistance control through at least one of joint driving assistance, torque compensation, trajectory amplification, mechanical limiting or damping, and lateral force compensation. Guide the patient to complete the preset rehabilitation training action.

[0026] The embodiment provides a limb movement disorder recognition and active motion assistance control method based on computer vision, which is applied to the active motion rehabilitation training scene of upper limbs and lower limbs. When the patient cannot complete the expected action due to movement disorder or insufficient muscle strength, in the process of active motion of upper limbs, abnormal shoulder joint shrugging, internal rotation or accompanied by obvious trunk compensation, elbow joint flexion and extension limitation or abnormal bending, wrist joint abnormal dorsiflexion, palmar flexion or end deviation from the target direction are prone to occur; in the process of active motion of lower limbs, when gait is limited, common manifestations include excessive swing or lifting of the hip joint, excessive forward or lateral inclination of the trunk, insufficient flexion and extension of the knee joint, knee extension or angular velocity stall, and ankle foot drop, excessive plantar flexion, inversion and other manifestations. The embodiment unifies visual perception, recognition and control linkage for the above abnormal joint manifestations, aiming to maximize the patient's active participation while ensuring training safety.

[0027] In the overall process of the embodiment, first, the image sequence of the upper and / or lower limbs of the patient during active movement is acquired through computer vision, and the three-dimensional coordinates of the trunk point, shoulder joint, elbow joint, wrist joint, and hip joint, knee joint, ankle joint, and toe point in the world coordinate system are extracted through image processing and pose estimation algorithm to construct time series data containing the spatial positions of multiple joints. Subsequently, the joint angle and joint angular velocity of each joint are calculated through the time series change of the joint position to form joint motion state information describing the joint motion process; in combination with the initial position, target position, and preset joint kinematics model, the expected angle change trend of each joint when completing a certain training task is given, and compared with the observed joint angle and joint angular velocity in real time to determine the current movement intention of the patient and whether there is movement disorder. According to the type and severity of the disorder, the assistance output of the rehabilitation robot or exoskeleton is further adjusted in the embodiment, so that the assistance strength and intervention joint change in real time with the patient's state.

[0028] In the aspect of abnormal recognition of upper limb active movement, the embodiment sets a visual sense recognition process for analyzing specific patterns of key joints such as shoulders, elbows, and wrists. When the three-dimensional coordinates of the shoulder joint region and the relative position relationship with the trunk point show that the shoulder is abnormally raised and the internal rotation angle is too large, and the trunk point is offset in the vertical or front-back direction beyond the normal range, it is considered that the shoulder is raised or the trunk is compensated. When the angle change of the elbow joint in the flexion and extension direction is obviously lagging behind the expected trajectory, or the angle change direction is opposite to the expected direction, it is considered that there is flexion and extension limitation or abnormal bending. When the angle of the wrist joint relative to the forearm deviates from the natural neutral position by a large margin, or the pointing direction of the end handle deviates from the target direction by an angle exceeding the preset threshold, it is considered that there is abnormal dorsiflexion, palmar flexion, or deviation from the target direction. Through the above recognition, the embodiment can distinguish between "real insufficient ability" and "forced to complete the action through compensatory posture" among the joints of the upper limb.

[0029] In order to improve the stability of the key feature extraction of the upper limb, the embodiment sets a pre-preprocessing step in the upper limb detection. First, in the initial state of the patient being quiet and motionless, the three-dimensional coordinates of the trunk point, shoulder joint, elbow joint and wrist joint in the world coordinate system are calibrated as the reference for subsequent calculation. During the movement, the differential or filter is used for speed and angular velocity estimation on the change of the position of each joint with time, and the smooth speed and joint angular velocity curve is obtained. In geometric modeling, the upper arm vector is defined by the connection line from the shoulder joint to the elbow joint, the forearm vector is defined by the connection line from the elbow joint to the wrist joint, and the trunk vector is defined by the connection line from the trunk point to the shoulder joint. The included angle between these vectors is calculated to obtain the joint angles of the shoulder joint, elbow joint and the like; in order to avoid abnormal noise interference, the embodiment sets an effective angle range for each joint angle, when the calculated angle exceeds the range, the output angle value fails or is unusable, and the abnormal value is not directly used in the subsequent obstacle identification, but is comprehensively judged in combination with the time neighborhood information and other joint data.

[0030] In the aspect of abnormal identification of active movement of the lower limb, the embodiment also identifies the joint through the visual sensing process, analyzes the hip joint, knee joint, ankle joint and trunk posture jointly. When it is observed that the swing amplitude of the hip joint obviously exceeds the preset range in the gait cycle, or the relative trajectory of the pelvis point shows that the hip joint is abnormally lifted, and the trunk point appears excessive forward inclination or lateral inclination, it is considered that the hip joint and the trunk posture are abnormal; when the flexion angle of the knee joint in the swing period always fails to reach the expected range, or the knee joint angle exceeds the normal straightening upper limit in the support period, and even gradually bends backward, it is determined that the knee joint is insufficient or the knee joint is reversed. When the knee joint angular velocity sequence appears frequent sudden reduction, stagnation or high frequency tremor mode, it is determined that the knee joint is stuck. For the ankle joint, when the foot segment is continuously observed in the swing period in the foot tip drooping posture and the dorsiflexion angle is insufficient, it is determined that the foot drop is determined; when the metatarsal flexion angle exceeds the threshold value for a long time in the weight-bearing or propulsion period, it is determined that the metatarsal flexion is excessive; when the foot segment is continuously deviated to the medial side or lateral side relative to the ground normal and exceeds the varus and valgus threshold value, it is determined that the ankle joint varus and valgus is abnormal.

[0031] In the pre-processing of lower limb detection, the embodiment first calibrates the three-dimensional coordinates of the hip joint, knee joint, ankle joint, toe point and pelvic point (as the trunk point) in the world coordinate system in the initial state of the patient without movement, and takes this initial posture as the reference benchmark for subsequent gait analysis. During the movement, the velocity and angular velocity of each joint position of the lower limb are estimated by difference or filter to suppress noise interference. In terms of geometric modeling, the thigh (upper leg) vector is defined by the line connecting the hip joint to the knee joint, the lower leg (lower leg) vector is defined by the line connecting the knee joint to the ankle joint, and the foot vector is defined by the line connecting the ankle joint to the toe point. The included angle between the above vectors is calculated to obtain the joint angles of the hip joint, knee joint and ankle joint at each stage, and the effective range of these angles is also set. When the angle exceeds the range, the angle value fails or the unusable flag is output. The above lower limb joint angles and corresponding angular velocities and three-dimensional position changes are jointly used as the basic data of lower limb joint motion state, which provides reliable basis for motion disorder recognition and auxiliary decision-making in gait.

[0032] In terms of judging whether the patient needs assistance and the assistance strength, the embodiment sets up a motion disorder auxiliary decision-making process, and cooperates with the force sensor to obtain force and moment support evidence. Through comprehensive analysis of joint angle, joint angular velocity and force near the joint, on the one hand, it is judged whether the patient encounters motion disorder in the current action, and on the other hand, it is evaluated whether there is obvious compensation mode. Based on these information, the embodiment calculates a compensation index to quantify the degree of compensation, and calculates the intention strength according to the situation of the patient trying to actively complete the action, and then forms the motion disorder recognition result containing "intention strength and whether there is compensation". When the result shows that the patient indeed has difficulty in completing the action by himself and the compensation risk is high, the embodiment starts or enhances the corresponding motion assistance output; when the intention strength is high but the compensation index is low, the assistance degree is appropriately reduced to encourage the patient to actively participate in training.

[0033] In the aspect of assisting the active movement of the upper limbs, the embodiment designs targeted control strategies for different abnormal behaviors of the joints. When the shoulder joint is detected to have abnormal behaviors such as hunching, internal rotation or accompanied by trunk compensation, this is regarded as the patient trying to compensate for the insufficient real joint movement ability by lifting the shoulder or swinging the trunk; in this case, instead of simply increasing the assisting force in the shoulder direction, the embodiment transfers the assisting output related to the shoulder to the elbow joint flexion and extension direction or the spatial movement direction of the forearm end, guiding the patient to complete the task with more reasonable elbow joint movement. When the elbow joint flexion and extension is limited or abnormal bending occurs, by comparing the angle between the force direction applied by the patient and the expected target direction, the compliance along the target direction is automatically increased and the resultant force deviating from the trajectory direction is reduced, so that the elbow joint can complete the flexion and extension movement near a safer and more natural trajectory. When the wrist joint has abnormal behaviors such as dorsiflexion, palmar flexion or the end direction deviates from the target, the embodiment applies a small correction torque at the end handle according to the visual detection of the abnormal angle of the wrist joint, gradually guiding the wrist joint to approach the natural neutral position, ensuring that the pointing direction of the handle is more stably towards the target direction, thereby improving the accuracy and comfort of upper limb training.

[0034] In the aspect of assisting the movement of the lower limbs, the embodiment gives differentiated intervention methods for typical problems such as insufficient flexion and extension of the knee joint, hyperextension of the knee joint and stiffness of the knee joint. When it is determined that the knee joint flexion angle is insufficient during the swing phase, the embodiment sets the knee joint as the key assisted object, applies an assisting torque along the knee joint flexion direction during the swing phase, and amplifies the knee joint flexion amplitude on the preset gait trajectory, guiding the patient to complete a larger angle flexion, thereby improving the step length and gait fluency; when it is determined that there is hyperextension of the knee joint during the standing or weight-bearing phase, a restraining or inhibiting assisting torque along the flexion direction is applied to the knee joint to limit excessive straightening and backward bending of the knee joint, thereby avoiding long-term damage to the joint structure; when it is observed that the knee joint angular velocity is stuck or tremors during consecutive gait cycles, the embodiment applies a rhythmic and small-amplitude flexion and extension assisting torque to the knee joint, which plays a role of “rhythm guidance”, prompting the knee joint angle change to be more continuous and smooth, and improving the overall gait coordination.

[0035] In terms of auxiliary measures for ankle movement disorders, the embodiment also gives specific control strategies for foot drop, excessive plantar flexion and varus and valgus. When it is identified that the foot drop is caused by insufficient dorsiflexion of the foot during the swing phase and the toes are drooping, the embodiment applies an auxiliary driving torque in the dorsiflexion direction of the ankle joint in this phase to lift the foot segment and reduce the risk of dragging and tripping; when it is identified that there is excessive plantar flexion of the ankle joint during standing or gait advancement, a mechanical limit or equivalent damping is set in the plantar flexion direction, and a slight dorsiflexion support is provided when weight-bearing to maintain the ankle joint in a more stable weight-bearing posture; when it is identified that the ankle joint has abnormal varus or valgus, the embodiment can provide support and limitation in the varus or valgus direction in combination with the multi-degree-of-freedom ankle exoskeleton structure, and cooperate with the lateral support force applied to the foot bottom to compensate for the lateral force, so that the contact between the foot bottom and the ground is more balanced, thereby improving the stability and safety during walking. Finally, by recording the movement disorder identification results, compensation indexes and auxiliary outputs during the training process, and providing real-time feedback through the human-computer interaction interface, the embodiment not only helps the patient to adjust the wrong posture in time and avoid movement injuries caused by improper operation, but also provides a basis for subsequent rehabilitation program optimization and individualized parameter adjustment, and realizes closed-loop management of the whole rehabilitation training process.

[0036] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0037] The principles and implementation modes of the present application are described by applying specific examples in this paper. The above description of the embodiments is only used to help understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed. In conclusion, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method for limb movement disorder recognition and active movement assistance control based on computer vision, characterized in that, include: Acquire motion image sequences of the patient's upper and / or lower limbs, and extract the three-dimensional coordinates of the trunk, shoulder joint, elbow joint, wrist joint, hip joint, knee joint, ankle joint and toe points to form spatial position data of multiple joints; Based on the spatial location data, construct multi-segment vectors of the limbs and calculate the vector angles to obtain joint angles and joint angular velocities, thus forming joint motion state data; Based on the joint motion state data and combined with force and torque support evidence, the joint angle deviation, angular velocity abnormality and torque abnormality are judged, the corresponding movement disorder type is identified, and it is determined whether there is trunk compensation or joint compensation behavior. The intention intensity and compensation index are output to form the movement disorder identification result. Based on the movement disorder identification results, the direction and magnitude of the target auxiliary joint and the auxiliary torque are determined, movement assistance control commands are generated, and active movement assistance control is implemented through at least one of the following methods: joint drive assistance, torque compensation, trajectory amplification, mechanical limit or damping, and lateral force compensation, to guide the patient to complete preset rehabilitation training movements.

2. The method for limb movement disorder recognition and active movement assistance control based on computer vision according to claim 1, characterized in that, Based on the spatial location data, construct multi-segment vectors of the limbs and calculate the angle between the vectors, including: Using the three-dimensional coordinates of the shoulder joint, elbow joint, wrist joint and torso point in the spatial location data, construct the upper arm vector from the shoulder joint to the elbow joint, the forearm vector from the elbow joint to the wrist joint and the torso vector from the torso point to the shoulder joint. Calculate the angle between the upper arm vector and the torso vector, and the angle between the upper arm vector and the forearm vector, to obtain the shoulder joint angle and the elbow joint angle; Based on the changes in the shoulder joint angle and elbow joint angle over time, the shoulder joint angular velocity and elbow joint angular velocity are obtained respectively, so that the joint motion state data includes at least the shoulder joint angle, shoulder joint angular velocity, elbow joint angle, and elbow joint angular velocity.

3. The method for limb movement disorder recognition and active movement assistance control based on computer vision according to claim 1, characterized in that, Constructing multi-segment vectors of limbs based on the spatial location data and calculating the angle between the vectors, further includes: Using the three-dimensional coordinates of the hip joint, knee joint, ankle joint, and toe points in the spatial location data, construct the thigh vector from the hip joint to the knee joint, the lower leg vector from the knee joint to the ankle joint, and the foot segment vector from the ankle joint to the toe points; Calculate the angle between the thigh vector and the calf vector, and the angle between the calf vector and the foot segment vector to obtain the knee joint angle and the ankle joint angle. Based on the changes of the knee joint angle and the ankle joint angle over time, obtain the knee joint angular velocity and the ankle joint angular velocity, respectively. The hip joint angle and hip joint angular velocity are determined based on the positional relationship of the hip joint in the spatial position data, so that the joint motion state data includes at least the hip joint angle, hip joint angular velocity, knee joint angle, knee joint angular velocity, ankle joint angle, and ankle joint angular velocity.

4. The method for limb movement disorder recognition and active movement assistance control based on computer vision according to claim 1, characterized in that, Based on the joint motion data and combined with force and torque support evidence, joint angle deviation, angular velocity abnormality, and torque abnormality are determined, and the corresponding movement disorder type is identified, including: In the joint motion state data, when the shoulder joint angle and shoulder joint angular velocity are inconsistent with the preset shoulder joint motion range and change pattern, and the displacement of the trunk point in the spatial position data determines that the trunk exceeds the preset range in the vertical or forward and backward directions, it is determined that there is a compensatory behavior of the trunk such as shrugging or the trunk leaning forward or bending to the side. In the joint motion state data, when the elbow joint angle does not reach the expected range within a preset time, or the elbow joint angle change trend is opposite to the preset target direction, and the elbow joint angular velocity shows a significant decrease or sudden change, the movement disorder of elbow joint flexion and extension or abnormal bending is determined by combining the force and torque support evidence. In the joint motion state data, when the joint angle of the wrist joint relative to the forearm continuously exceeds the preset normal range, or when the wrist movement trajectory obtained according to the spatial position data deviates from the target direction by more than a preset threshold, it is determined that there is a movement disorder of abnormal dorsiflexion, palmar flexion or deviation of the wrist joint.

5. The method for limb movement disorder recognition and active movement assistance control based on computer vision according to claim 1, characterized in that, Based on the joint motion state data, and combined with force and torque support evidence, the system determines joint angle deviation, angular velocity abnormality, and torque abnormality, identifies the corresponding type of movement disorder, and further includes: During the gait swing phase, if the maximum flexion value of the knee joint angle in the joint motion state data is lower than the preset flexion threshold, it is determined that there is insufficient knee flexion and extension. During the gait support phase, if the knee joint angle in the joint motion state data exceeds the preset upper limit of extension and shows a backward bending trend, and the force and torque support evidence related to the knee joint shows abnormal force, it is determined that there is a knee retraction. When the knee joint angular velocity in the joint motion state data shows frequent pauses, abrupt changes, or irregular high-frequency fluctuations within a continuous gait cycle, it is determined that there is a movement disorder involving knee joint locking.

6. The method for limb movement disorder recognition and active movement assistance control based on computer vision according to claim 1, characterized in that, Based on the joint motion state data, and combined with force and torque support evidence, the system determines joint angle deviation, angular velocity abnormality, and torque abnormality, identifies the corresponding type of movement disorder, and further includes: During the gait swing phase, if the ankle joint angle in the joint motion state data does not reach the preset dorsiflexion angle, and the foot segment is determined to be in a toe-down posture based on the spatial position data, foot drop is determined to exist. During standing or gait progression, if the ankle joint angle in the plantar flexion direction in the joint motion state data continuously exceeds a preset range, and the force and torque support evidence related to the ankle joint indicates instability in weight-bearing, it is determined that there is excessive plantar flexion of the ankle joint. When the foot segment is determined to deviate continuously from the direction of inversion or eversion relative to the ground normal direction based on the spatial position data and exceeds a preset range, it is determined that there is a movement disorder of ankle inversion or eversion.

7. The method for limb movement disorder recognition and active movement assistance control based on computer vision according to claim 1, characterized in that, Based on the movement disorder identification results, the target assist joint and the direction and magnitude of the assist torque are determined, movement assist control commands are generated, and active movement assist control is implemented through at least one of the following methods: joint drive assistance, torque compensation, trajectory amplification, mechanical limiting or damping, and lateral force compensation, to guide the patient to complete preset rehabilitation training movements, including: When the movement disorder identification results indicate the presence of shoulder shrugging or trunk compensation behavior, reduce the auxiliary force along the trunk lifting or forward leaning direction, and adjust the auxiliary force to target the elbow joint as the auxiliary joint and apply auxiliary torque along the elbow joint flexion and extension direction; When the movement disorder identification result indicates that there is limited flexion and extension or abnormal bending of the elbow joint, the elbow joint is set as the target auxiliary joint. The direction and magnitude of the auxiliary torque are determined according to the deviation between the elbow joint angle and the elbow joint angular velocity and the preset target, so that the elbow joint movement approaches the preset target trajectory. When the movement disorder identification result indicates the presence of abnormal dorsiflexion or palmar flexion of the wrist joint, the wrist joint is set as the target auxiliary joint, and an auxiliary torque is applied at the end-effector position to counteract the abnormal dorsiflexion or palmar flexion, so that the wrist joint is kept in a near-neutral position.

8. The method for limb movement disorder recognition and active movement assistance control based on computer vision according to claim 1, characterized in that, Based on the movement disorder identification results, the target assist joint and the direction and magnitude of the assist torque are determined, movement assist control commands are generated, and active movement assist control is implemented through at least one of the following methods: joint drive assistance, torque compensation, trajectory amplification, mechanical limiting or damping, and lateral force compensation, to guide the patient to complete preset rehabilitation training movements. This also includes: When the movement disorder identification result indicates that there is insufficient knee flexion and extension, the knee joint is set as the target auxiliary joint, and an auxiliary torque is applied along the knee flexion direction during the gait swing phase, and the knee flexion angle is amplified within a preset range. When the movement disorder identification result indicates the presence of hyperextension of the knee, the knee joint is set as the target auxiliary joint, and a limiting auxiliary torque is applied along the knee flexion direction during the gait support phase to suppress hyperextension and hyperbending of the knee joint. When the movement disorder identification results indicate the presence of knee joint slack, rhythmic, small-amplitude flexion-extension auxiliary torques are applied to the knee joint during continuous gait cycles to improve the continuity and smoothness of knee joint angle and knee joint angular velocity.

9. The method for limb movement disorder recognition and active movement assistance control based on computer vision according to claim 1, characterized in that, Based on the movement disorder identification results, the target assist joint and the direction and magnitude of the assist torque are determined, movement assist control commands are generated, and active movement assist control is implemented through at least one of the following methods: joint drive assistance, torque compensation, trajectory amplification, mechanical limiting or damping, and lateral force compensation, to guide the patient to complete preset rehabilitation training movements. This also includes: When the movement disorder identification result indicates the presence of foot drop, the ankle joint is set as the target auxiliary joint, and an auxiliary torque is applied along the dorsiflexion direction of the ankle joint during the gait swing phase to lift the foot and reduce the risk of dragging. When the movement disorder identification result indicates that there is excessive plantar flexion of the ankle joint, the ankle joint is set as the target auxiliary joint, an upper limit of the auxiliary torque or equivalent damping is set in the plantar flexion direction, and support is provided in the dorsiflexion direction of the ankle joint when weight-bearing is required; When the movement disorder identification result indicates the presence of ankle inversion or eversion, the ankle joint is designated as the target auxiliary joint, and a lateral auxiliary force or limiting auxiliary torque is applied in the inversion or eversion direction to maintain balanced contact of the sole of the foot.

10. The method for limb movement disorder recognition and active movement assistance control based on computer vision according to claim 1, characterized in that, Also includes: During multiple rehabilitation training sessions, the intent intensity, compensation index, and corresponding motor assist control commands in the motor impairment recognition results obtained from each training session are recorded. In subsequent training sessions, the selection of the target assist joint and the direction and magnitude of the assist torque are adaptively adjusted based on the recorded content to optimize the active motor assist control effect.

Images