A rehabilitation training evaluation method based on posture recognition

Abstract

The application discloses a rehabilitation training evaluation method based on posture recognition, and relates to the technical field of computer vision, which comprises the following steps: collecting a video sequence of a trainer performing a rehabilitation training action; performing human posture recognition on the video sequence to obtain a skeletal key point sequence of the trainer; and taking the head key point in the skeletal key point sequence as the basis. The application can make the evaluation result truly reflect the action quality of the trainer, solve the problem of distorted evaluation result in the background technology, and improve the training completion rate of the cognitively impaired elderly trainer. Moreover, the application does not need to add additional sensors such as electroencephalogram and electromyogram, and can be conveniently deployed in a home scene.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically to a rehabilitation training assessment method based on posture recognition. Background Technology

[0002] As the population ages, the number of elderly people with cognitive impairment continues to grow. Continuous rehabilitation training for this population is crucial for maintaining their physical function and slowing down functional decline. A posture recognition-based rehabilitation training assessment system provides an automated approach: it captures videos of trainees performing training movements using a camera, extracts key skeletal points using a posture recognition model, and compares the trainee's movements with a standard movement template to obtain assessment results. This allows elderly trainees to independently conduct rehabilitation training and receive feedback on movement quality even in environments without therapist supervision, such as at home or in the community.

[0003] However, the inventors discovered in practice that when the above system is applied to elderly trainees with cognitive impairment, the assessment results are distorted: when elderly trainees with cognitive impairment perform training tasks independently in an unattended environment, they may forget the training task and leave the training area midway, or perform other actions unrelated to the training movements. As a result, the posture sequences collected by the assessment system contain segments in which the trainees have not performed the training movements. When the assessment system evaluates the above sequences as a whole, these segments are included in the assessment criteria. Even when using a method of filtering data based on the range of motion, large-amplitude movements caused by the trainee leaving midway or moving around randomly are judged as valid data and retained, making it impossible to remove these segments. Therefore, the assessment results output by the assessment system cannot truly reflect the quality of the trainee's movements, and the adjustments made to the rehabilitation program based on the assessment results also lose their reliable basis. Summary of the Invention

[0004] The purpose of this invention is to provide a rehabilitation training assessment method based on posture recognition to address the shortcomings of the prior art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a rehabilitation training assessment method based on posture recognition, comprising:

[0006] Collect video sequences of trainees performing rehabilitation training movements;

[0007] Human pose recognition is performed on the video sequence to obtain the skeletal key point sequence of the trainee;

[0008] Based on the head key points in the skeletal key point sequence, the stability of the trainee's head posture within the sliding time window is determined, and the correlation between the skeletal key point sequence corresponding to the target joint and the action sequence corresponding to the standard action template within the sliding time window is determined. Continuous time periods in which the stability is greater than a preset first threshold and the correlation is greater than a preset second threshold are determined as effective action segments.

[0009] The effective movement segments are compared with the standard movement template to obtain the evaluation results of the rehabilitation training movements.

[0010] In a preferred embodiment, determining the stability of the trainee's head posture within a sliding time window includes:

[0011] Based on the aforementioned head key points, calculate the yaw angle and pitch angle of the trainee's head;

[0012] Calculate the standard deviation of the yaw angle and the pitch angle within the sliding time window, and determine the stability as the reciprocal of the standard deviation;

[0013] The method further includes: in response to the angle between the trainee's head orientation and a preset reference direction being greater than a preset angle threshold, determining the corresponding time period as a segment in which the trainee did not cooperate in performing the rehabilitation training action.

[0014] In a preferred embodiment, determining the stability of the trainee's head posture within a sliding time window includes:

[0015] In response to the confidence level of the head key points in the skeletal key point sequence being lower than a preset confidence threshold, the change in the deviation angle of the trainee's body orientation within the sliding time window is determined based on the trunk key points in the skeletal key point sequence, and the displacement velocity of the trainee's center of mass within the sliding time window is determined based on the pelvic key points in the skeletal key point sequence.

[0016] The stability is determined based on the change in the deviation angle and the displacement velocity of the center of mass.

[0017] In a preferred embodiment, determining the correlation between the skeletal keypoint sequence corresponding to the target joint within the sliding time window and the action sequence corresponding to the standard action template includes:

[0018] Calculate the joint angle sequence of the target joint within the sliding time window based on the skeletal key point sequence corresponding to the target joint;

[0019] Calculate the normalized cross-correlation coefficient between the joint angle sequence and the standard angle sequence corresponding to the standard motion template;

[0020] The normalized cross-correlation coefficient is determined as the correlation degree.

[0021] In a preferred embodiment, determining the continuous time period in which the stability is greater than a preset first threshold and the correlation is greater than a preset second threshold as a valid action segment includes:

[0022] In response to the stability within the sliding time window being greater than the preset first threshold and the correlation increasing from less than the preset second threshold to greater than the preset third threshold, the sliding time window is marked as the start of the effective action segment;

[0023] In response to the stability within the sliding time window being greater than the preset first threshold and the correlation decreasing from greater than the preset second threshold to less than the preset fourth threshold, the sliding time window is marked as the end of the effective action segment;

[0024] Wherein, the preset third threshold is greater than the preset second threshold, and the preset fourth threshold is less than the preset second threshold.

[0025] In a preferred embodiment, the method further includes:

[0026] In response to the loss of the skeletal key points corresponding to the target joint in consecutive image frames, or the removal of the trainee's body area from the captured image of the video sequence for more than a first preset duration, the corresponding segment is determined to be a segment in which the trainee did not cooperate in performing the rehabilitation training action.

[0027] In addition, in response to the stability being greater than the preset first threshold and the correlation being continuously less than or equal to the preset second threshold for more than a second preset duration, the corresponding continuous time period is determined as a segment in which the trainee does not cooperate in performing the rehabilitation training action.

[0028] In a preferred embodiment, determining the continuous time period in which the stability is greater than a preset first threshold and the correlation is greater than a preset second threshold as a valid action segment further includes:

[0029] Remove valid action segments with a duration shorter than the preset fragment duration, and merge adjacent valid action segments with an interval shorter than the preset interval duration;

[0030] Furthermore, the evaluation result includes a confidence level, which is the proportion of the total duration of the effective action segment to the total duration of the video sequence.

[0031] In a preferred embodiment, the method further includes:

[0032] If a segment in which the trainee fails to cooperate in performing the rehabilitation training movements continues for more than a third preset duration, the assessment timer is paused and a prompt message guiding the trainee to resume cooperation is output.

[0033] In addition, in response to the recurrence of consecutive time periods in which the stability is greater than the preset first threshold and the correlation is greater than the preset second threshold, the evaluation timing is resumed.

[0034] In a preferred embodiment, comparing the effective movement segment with the standard movement template to obtain the evaluation result of the rehabilitation training movement includes:

[0035] Based on the stability and correlation within the effective action segment, determine the coordination weight corresponding to the effective action segment;

[0036] The comparison results of the effective action segments are weighted using the aforementioned matching weights to obtain the evaluation result.

[0037] In a preferred embodiment, the method further includes:

[0038] Select effective movement segments from the trainee's historical training performance that have an evaluation result better than the preset evaluation level.

[0039] The standard motion template is updated with the skeletal keypoint sequence corresponding to the selected effective motion segment.

[0040] The technical effects and advantages provided by the present invention in the above technical solution are as follows:

[0041] This method enables the assessment results to accurately reflect the quality of the trainee's movements, resolving the issue of distorted assessment results in the previous techniques. The assessment results are stable and reliable, and will not be repeatedly broken due to small fluctuations in the judgment boundaries, and their reliability can be quantified. The assessment benchmark is automatically adjusted according to the trainee's rehabilitation progress, so that the assessment results are consistent with the trainee's own rehabilitation level, achieving the effect of individualized assessment. In addition, trainees can be promptly guided back to training when they have not cooperated for a long time, improving the training completion rate of elderly trainees with cognitive impairment. Furthermore, this method does not require additional sensors such as EEG or EMG, and can be conveniently deployed in home settings. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0043] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] Application Overview:

[0046] Existing posture recognition-based rehabilitation training assessment schemes can be broadly categorized into three types based on how they handle the trainee's cooperation status: The first type is attention monitoring schemes, which output the trainee's level of attention through head posture estimation, gaze estimation, etc. However, this type of scheme only outputs attention as an independent monitoring result and does not intervene in the data link upon which the assessment is based. The second type is effective frame screening schemes, which filter data based on the amplitude of motion between adjacent image frames (such as the area of ​​contour difference). However, elderly trainees with cognitive impairment who leave the training area midway and wander around generate large-amplitude movements, and this type of scheme will instead retain such non-cooperative segments as valid data. The third type is template comparison assessment schemes, which compare the collected complete sequence with a standard action template as a whole, assuming that the trainee is in a cooperative state throughout the process, and do not remove any non-cooperative segments.

[0047] Therefore, it is evident that none of the three aforementioned solutions address the following situation: segments where the trainee did not perform the training movements are mixed into the posture sequence and included in the evaluation criteria, leading to distorted evaluation results. To address this issue, the basic concept of this application is: before performing template comparison evaluation on the posture sequence, the posture sequence is first screened online based on the trainee's cooperation status—the stability of head posture characterizes whether the trainee is focused on the training task, and the correlation between the posture sequence and the standard movement template characterizes whether the trainee is performing the training movement. Only continuous periods where both conditions are simultaneously met are considered valid movement segments for evaluation, thereby cutting off the path of contamination of the evaluation results by uncooperative segments.

[0048] It should be noted that although the embodiments of this application will be described using home rehabilitation training for elderly trainees with cognitive impairment as an example, the above basic concept of this application can also be applied to other trainee groups with difficulties in cooperation, such as children, people with attention deficit, and patients under postoperative analgesia. As long as there is a situation where there is non-cooperation in the assessment data, the dual-gating screening concept of this application can be adopted.

[0049] Exemplary method:

[0050] Figure 1 The illustration shows a flowchart of a posture recognition-based rehabilitation training assessment method according to an embodiment of this application.

[0051] like Figure 1 As shown, the method includes steps S1 to S4. In step S1, a video sequence of the trainee performing rehabilitation training movements is acquired. This step is the data source for all subsequent steps: only after obtaining a continuous video sequence can image frames available for posture recognition processing exist. Taking a home scenario as an example, the video sequence can be acquired by the front-facing camera of a smart TV, tablet, or mobile phone at a rate of 24 to 60 frames per second (preferably 30 frames per second); the camera can be positioned facing the training area, with its optical axis forming an angle of no more than 30° with the coronal plane of the trainee's torso, and the shooting distance can be 1.5 meters to 3 meters, so that the trainee's entire body is in the frame. The purpose of this step is to obtain the trainee's training process data in a non-contact manner, which allows the trainee to start training without wearing any sensors, lowering the barrier to home deployment.

[0052] After acquiring the video sequence in step S1, step S2 is executed: human pose recognition is performed on the video sequence to obtain the skeletal key point sequence of the trainee.

[0053] In some implementations, pose recognition can employ a deep learning-based human pose estimation model (such as a model that outputs 17 or 33 human keypoints). For each frame, the model outputs the two-dimensional coordinates of each keypoint and its corresponding confidence score. The confidence score characterizes the reliability of the model's localization result for that keypoint, ranging from 0 to 1. The set of keypoint coordinates and confidence scores arranged in frame order constitutes the skeletal keypoint sequence. This step converts unstructured video into a structured, computable coordinate sequence, providing a computable judgment object for the gating decision in step S3. Optionally, pose recognition can also use a depth camera to obtain three-dimensional keypoint coordinates, or supplement keypoints with inertial measurement unit data from areas such as the wrist and waist. This application does not limit the specific implementation method of pose recognition.

[0054] After obtaining the skeletal key point sequence in step S2, step S3 is executed: determine the stability of the trainee's head posture within the sliding time window, determine the correlation between the skeletal key point sequence corresponding to the target joint and the action sequence corresponding to the standard action template, and determine the continuous time period with stability greater than a preset first threshold and correlation greater than a preset second threshold as the effective action segment.

[0055] Step S3 may further include steps S31 to S33.

[0056] In step S31, the stability of the trainee's head posture within the sliding time window is determined based on the head key points in the skeletal key point sequence; in step S32, the correlation between the skeletal key point sequence corresponding to the target joint and the action sequence corresponding to the standard action template within the sliding time window is determined; wherein, steps S31 and S32 can be executed in parallel or sequentially, and the two are independent of each other.

[0057] In step S33, in response to the results of steps S31 and S32, a continuous time period with stability greater than a preset first threshold and correlation greater than a preset second threshold is determined as an effective action segment, i.e., satisfying the condition shown in equation (1):

[0058] ;

[0059] In equation (1), VTW is the set of effective action segments, [ , [ ] represents the start and end times of the effective action segment, S represents the stability, and ρ represents the correlation. The first threshold, This is the second threshold. The purpose of this step (S3) is to perform online purification of the pose sequence using the trainer's cooperation status as a gate, so that uncooperative segments no longer enter the data link for subsequent evaluation.

[0060] The reason for using steps S31 and S32 in combination is that using only either step would result in misjudgment among the target population of this application.

[0061] When using only the stability gate, a trainee might keep their head steadily focused on the screen but perform movements unrelated to the training (the person is watching, but the movement is incorrect), which cannot be identified. When using only the relevance gate, if the trainee performs movements that are partially similar to the training movements but are essentially unrelated, it is impossible to distinguish whether the movement is improper or if the trainee is not performing the training task at all. Only by combining both gates can "non-cooperation" be separated from "cooperation but poor movement quality," ensuring that the filtered-out objects are precisely the non-cooperation segments rather than the segments with poor movement quality—this is the essential difference between the concept of this application and solutions that only monitor attention or only screen movement range.

[0062] After obtaining the effective movement segment in step S3, step S4 is executed: the effective movement segment is compared with the standard movement template to obtain the evaluation result of the rehabilitation training movement.

[0063] Step S4 may further include:

[0064] Step S41: Align the effective action segment with the standard action template. The alignment can be achieved using dynamic time warping, which is an algorithm that allows two time series of different lengths and rhythms to be non-linearly aligned according to the minimum cumulative distance, thereby eliminating the influence of the trainee's action speed difference on the deviation calculation.

[0065] Step S42: After alignment, calculate the joint-level deviation for each joint, as shown in Equation (2):

[0066] ;

[0067] In equation (2), Let be the angle deviation of the j-th target joint at time t. For the trainee's joint angles. The joint angle is the corresponding joint angle after alignment mapping τ of the standard motion template. The joint angle can be calculated from the two bone segment vectors constituting the joint, as shown in equation (3):

[0068] ;

[0069] In equation (3), , These are the vectors of the two bone segments constituting the target joint, determined by the coordinate difference of key points on adjacent bones. In step S43, the mean square deviation of the effective movement segments is obtained based on the statistical analysis of deviations at each joint level, thus yielding the evaluation result of this training. The purpose of this step (S4) is to transform the purified data into quantifiable evaluation conclusions; since the input of step S4 only comes from the effective movement segments filtered out in step S3, the evaluation result no longer includes the contribution of non-coordinated segments, thereby truly reflecting the quality of the trainee's movements.

[0070] Steps S1 to S4 are executed sequentially and are interconnected: S1 provides raw data, S2 structures the data, S3 cleans the structured data, and S4 outputs evaluation results based on the cleaned data. The four steps constitute a complete technical solution.

[0071] Data Preprocessing and Sliding Time Window: In some implementations, after obtaining the skeletal keypoint sequence in step S2 and before the gating decision in step S3, the skeletal keypoint sequence can be preprocessed to reduce the interference of recognition noise on the gating decision. Preprocessing may include: applying median filtering or moving average filtering to the keypoint coordinates between adjacent frames to suppress single-frame jitter; for isolated frames where the confidence level is lower than a preset confidence threshold due to brief occlusion, the keypoint coordinates are supplemented by linear interpolation of adjacent valid frames, but periods with consecutive missing frames exceeding a preset number (e.g., 10 consecutive frames) are not interpolated and are directly submitted to the gating decision; and normalizing the keypoint coordinates of each frame according to the length of the human torso to eliminate the influence of changes in the distance between the trainee and the camera on the coordinate scale. The reason for interpolating isolated frames but rejecting interpolation for consecutive missing segments is that consecutive missing segments often correspond to genuine non-cooperation states such as the trainee leaving the seat; forcibly supplementing them with interpolation would disguise the non-cooperation segments as valid data. The effect of this preprocessing step is to improve the stability and authenticity of the input data for subsequent gating decisions. Alternatively, time-series filtering methods such as Kalman filtering can also be used for filtering.

[0072] The sliding time window slides across the skeletal keypoint sequence with a fixed step size. The window length can be 0.5 to 1.0 times the standard action template action cycle, and the step size can be 1 frame to one-quarter of the window length. If the window length is too short, the statistical significance of stability and correlation will be insufficient, and if the window length is too long, it will delay the detection boundary of non-matching segments. Taking 30 frames per second acquisition and a template action cycle of 4 seconds as an example, the window length can be 2 to 4 seconds (60 to 120 frames), and the step size can be 10 to 30 frames. Each sliding time window outputs a pair of gating values ​​(stability S and correlation ρ) for use in the dual-gating judgment of Equation (1). The above configuration of window length and step size is to achieve a balance between judgment sensitivity and statistical stability, so that the gating can detect non-matching segments in a timely manner without frequent flipping due to single-frame noise.

[0073] Door 1: Determination of head posture stability; In one implementation, the head posture stability in step S31 is calculated based on head key points: First, the head posture vector can be formed by taking the key points of both ears and the key point of the nose tip, and the yaw angle and pitch angle of the head can be calculated; The reason for choosing the head key points is that the head orientation is the most direct visual representation of whether the trainee is looking at the training content. Then, the standard deviation σ of the yaw angle and pitch angle within the sliding time window is calculated, as shown in Equation (4):

[0074]

[0075] In equation (4), Let be the yaw angle or pitch angle of the i-th frame within the window. Let n be the mean within the window, and n be the number of frames within the window. The standard deviation is used instead of the mean to characterize the coordination state because the disruption of the coordination state is first manifested as small and frequent head movements (increased jitter), while the mean may not shift when looking symmetrically from left to right; the standard deviation is sensitive to jitter, and its effect is to capture signs of inattention and looking around earlier than the mean. Then, the reciprocal of the standard deviation is determined as the stability, as shown in equation (5):

[0076] ;

[0077] The reciprocal mapping of equation (5) makes "the greater the stability, the better the fit", thus aligning with the gating direction of "stability greater than the first threshold" in equation (1). This results in a more intuitive directionality for gating decisions, facilitating the setting and adjustment of the threshold. First threshold The standard deviation is given in terms of stability. When calibrated using standard deviation, the corresponding angle standard deviation threshold can be between 8° and 20°, preferably 12°. During normal screen viewing training, the angle standard deviation of head micro-movements usually does not exceed 10°, while the angle standard deviation when distracted or looking around usually exceeds 15°. Therefore, the benchmark is calibrated within this range. In addition, to further exclude cases where the trainee clearly turns their head away, in response to the angle between the head orientation and the preset reference direction (such as the screen normal direction) being greater than the preset angle threshold (which can be 45°), the corresponding time period can be directly judged as a segment where the trainee did not cooperate with the execution. The purpose of this additional judgment is to quickly intercept cases where the trainee clearly turns their head away. Its effect is to avoid missed judgments of large-angle head deviations that are not captured by the standard deviation in time due to statistical smoothing within the window.

[0078] In another implementation (corresponding to scenarios where the head is obscured or the elderly person is looking down), when the confidence of the head keypoints is lower than a preset confidence threshold (which can be between 0.5 and 0.7), stability can be determined using trunk and pelvic information: the trunk vector is constructed by the midpoints of the shoulder keypoints and the midpoints of the pelvic keypoints, and the change in the angle between the trunk vector and the vertical direction within the sliding time window is calculated (such as the standard deviation of the angle). The centroid displacement velocity is obtained by removing the horizontal position of the midpoint of the key point of the pelvis for the corresponding duration. After normalizing both, the coefficients are combined to form the substitution stability S′, as shown in equation (6):

[0079] ;

[0080] In equation (6), , The weighting coefficient (which can be 0.6 or 0.4) is consistent with the dimension of stability S in equation (5) (both are more consistent). It is also compared with the first threshold to participate in the dual-gating decision. The trigger condition for this degradation path is that the confidence of the head key point is lower than the preset confidence threshold. Its function is to provide a consistent state determination even in scenarios where the head key point is unreliable, such as when the elderly are looking down or sitting and occluding. Its effect is to prevent gate 1 from failing as a whole due to the failure of the head key point, thus ensuring the robustness of the dual-gating. The stability dimension of the output of the two paths is consistent, and they can be seamlessly switched at the frame level according to the confidence of the head key point.

[0081] Gate 2: Determining the relevance. In one implementation, the relevance in step S32 is calculated as follows:

[0082] Based on the skeletal keypoint sequence corresponding to the target joint, calculate the joint angle sequence of the target joint within the sliding time window according to Equation (3); calculate the normalized cross-correlation coefficient between the joint angle sequence and the standard angle sequence corresponding to the standard motion template, as shown in Equation (7), and determine the normalized cross-correlation coefficient as the correlation:

[0083] ;

[0084] In equation (7), Let be the joint angle of the i-th frame within the window. These are the corresponding angles in the standard angle sequence. The effect of using normalized cross-correlation coefficients is that they are insensitive to the overall scaling of movement amplitude, focusing instead on whether the trend of movement pattern changes is consistent with the template. Therefore, even if the trainee's movement amplitude is smaller due to insufficient muscle strength, as long as the movement pattern trend is correct, the correlation can still maintain a high level, and it will not be misjudged as non-cooperation due to amplitude differences. Second threshold. The value can be between 0.5 and 0.8, with 0.65 being preferred. Target joints can be configured according to the type of training movement: shoulder and elbow joints for upper limb movements, hip and knee joints for lower limb movements, and equivalent joints corresponding to the trunk tilt angle for balance training. The purpose of configuring target joints according to the type of movement is to focus the relevance on the joints targeted by the current training movement, which avoids interference from random movements of irrelevant joints in the judgment.

[0085] In another implementation, the correlation can also be determined by the reciprocal of the cumulative distance after dynamic time warping alignment, or the matching degree between the attitude state transition sequence and the standard state transition sequence. This application does not limit this. The above alternative methods are functionally equivalent to the normalized cross-correlation coefficient, both of which are measures that characterize the "similarity between the current action sequence and the template action sequence".

[0086] Hysteresis Gating: Stable Determination of Effective Action Segment Boundaries: In actual training, the trainee's relevance often fluctuates slightly around a second threshold. If the second threshold is used as the sole boundary for entering and exiting effective action segments, the jittering of relevance around the threshold will cause effective action segments to break repeatedly, contaminating subsequent evaluations.

[0087] Therefore, a hysteresis gate is introduced in one implementation, wherein the second threshold is the baseline judgment threshold, the third threshold is the correction threshold for entering the effective action segment, and the fourth threshold is the correction threshold for exiting the effective action segment: in response to the stability being greater than the first threshold and the correlation rising from less than the second threshold to greater than the third threshold within the sliding time window, the sliding time window is marked as the beginning of the effective action segment; in response to the stability being greater than the first threshold and the correlation falling from greater than the second threshold to less than the fourth threshold, the sliding time window is marked as the end of the effective action segment; the relationship between the thresholds is shown in Equation (8):

[0088] ;

[0089] In equation (8), Δ is the hysteresis bandwidth, which can be taken as 0.05 to 0.15, preferably 0.1. That is, the correlation must rise to a third threshold above the benchmark to enter the effective action segment, and must fall to a fourth threshold below the benchmark to exit the effective action segment. Fluctuations within the hysteresis band (from the fourth threshold to the third threshold) do not change the state of the effective action segment. Thus, the boundary of the effective action segment remains stable when the correlation fluctuates slightly. The effect is to avoid repeated breaks in the effective action segment and frequent start and stop of the evaluation time due to threshold boundary jitter, so that the data segment on which the evaluation is based is complete and continuous.

[0090] Contextualized Determination of Uncooperative Segments: For elderly trainees with cognitive impairment, uncooperative behavior exhibits two typical forms. The first is the "absent-from-the-seat" type: the trainee leaves the training area midway, manifested as the loss of key skeletal points corresponding to the target joint in consecutive image frames, or the trainee's body area moving out of the frame. If this state exceeds a first preset duration (1 to 3 seconds, preferably 2 seconds), the corresponding segment is determined as an uncooperative segment. The second is the "task-forgotten" type: the trainee does not leave and their head posture is stable (stability greater than a first threshold), but forgets the training task and performs irrelevant actions, manifested as a relevance consistently less than or equal to a second threshold. If this state exceeds a second preset duration (3 to 10 seconds, preferably 5 seconds), the corresponding continuous time period is determined as an uncooperative segment. These two determination branches can be executed in parallel with dual-gated determinations; explicitly marked segments are also excluded from evaluation.

[0091] From a behavioral perspective, the two aforementioned forms correspond to two typical pathological manifestations in elderly trainers with cognitive impairment: The apathy type stems from impaired task persistence and shifting attention away from the environment; the trainee's ability to maintain the behavioral goal of "performing a training task" is reduced, making them easily attracted by other stimuli in the environment and causing them to leave. The task forgetting type stems from impaired working memory; although the trainee maintains a screen-facing posture (formally still "participating"), they have forgotten the actions they were supposed to perform. These two forms are distinctly represented in their postural sequences—the former is characterized by the overall loss of key points or the body area being out of frame, while the latter is characterized by a stable head but unrelated movements—making them difficult to capture uniformly using a single gating method. This is precisely why this application further sets up a scenario-based judgment branch in addition to dual gating. The effect of this supplementary judgment is to explicitly capture the two most easily overlooked non-cooperational forms, apathy and task forgetting, further improving the purification rate.

[0092] Post-processing and confidence of valid action segments: After obtaining valid action segments, one implementation may also include:

[0093] Valid action segments with a duration shorter than the preset fragment duration (0.5 seconds) are removed, and adjacent valid action segments with an interval shorter than the preset interval duration (0.3 seconds) are merged. This removes residual spikes near the gating boundary, ensuring that the data segments entering the evaluation are temporally complete and continuous. The evaluation result includes a confidence level C, as shown in equation (9):

[0094] ;

[0095] In equation (9), K is the number of effective action segments. , These represent the end and start times of the k-th valid action segment, respectively. This represents the total duration of the video sequence. The confidence score characterizes the proportion of data used in this assessment relative to all collected data: when C is lower than a preset proportion (e.g., 0.3), it indicates that the assessment results are unreliable and suggests retraining. Outputting the confidence score allows therapists to judge the credibility of the assessment results, avoiding the misuse of assessment results with a low proportion of data for adjusting rehabilitation plans.

[0096] A closed loop of suspension, recall, and re-evaluation:

[0097] In one implementation, in response to a segment of uncooperative execution lasting longer than a third preset duration (e.g., 8 to 15 seconds, preferably 10 seconds), the assessment timing is paused and a prompt message guiding the trainee to resume cooperation is output; the prompt message can be a voice prompt, a highlighted training area on the screen, or a replay of a standard movement demonstration. Subsequently, in response to the recurrence of a continuous period with stability greater than a first threshold and relevance greater than a second threshold (preferably confirmed after one template movement cycle), the assessment timing is resumed. The triggering and deactivation conditions for this pause and resumption of assessment are based on the aforementioned dual-gating judgment results. The effect is that no invalid assessment time is generated during the uncooperative period, and the assessment continues seamlessly after being guided back, improving the training completion rate for elderly trainees with cognitive impairment.

[0098] The format of the prompts can be adapted according to the type of non-cooperation: for absenteeism, voice prompts (such as announcing the trainee's name and prompting them to return to the training area) and highlighting the border of the training area on the screen; for task forgetting, the standard demonstration video of the current training movement is replayed, possibly supplemented with a breakdown diagram of the movement. During the pause in the evaluation timer, video sequences continue to be collected and gating is performed, but no data is added to the evaluation results. Once the recovery conditions are met, the process resumes from the interruption point. The pause and recovery determination can also be combined with the aforementioned hysteresis gating execution, i.e., the recovery confirmation requires the relevance to rise above the third threshold and remain for a preset duration, to avoid the trainee lingering at the gating boundary, causing frequent start and stop of the evaluation timer.

[0099] Combined with weighted evaluation: Dual gating divides each time period into valid or invalid in a binary manner, but the degree of cooperation still varies among time periods within the valid action segment.

[0100] For effective action segments, the assessment criteria are more reliable for periods with higher stability and correlation.

[0101] To utilize this difference, in one implementation, the comparison to obtain the evaluation result in step S4 can be performed as follows: First, determine the coordination weight corresponding to the effective motion segment based on the stability and relevance within the effective motion segment, as shown in equation (10); wherein, the comparison result of the effective motion segment may include the segment score of the effective motion segment and the joint-level deviation of each target joint:

[0102] ;

[0103] In equation (10), The coordination weight for the k-th effective action segment. , These are the mean values ​​of stability and relevance within the segment (which can be normalized and then synthesized), respectively. α and β are weighting coefficients, and α+β=1, preferably α=0.4 and β=0.6. Subsequently, the comparison results of each effective action segment are weighted according to the matching weights to obtain the evaluation result, as shown in equation (11):

[0104] ;

[0105] In equation (11), Let Score be the segment score of the k-th valid action segment, and Score be the evaluation result of this training. After weighting, valid action segments with high cooperation contribute more to the evaluation result, while the influence of segments with cooperation near the gating boundary is smoothed and weakened. Therefore, the evaluation result is no longer sensitive to subtle fluctuations in the gating boundary. It should be noted that the cooperation weight has a different function than the confidence C mentioned above: the former describes the contribution distribution of cooperation quality within each valid segment to the evaluation result, while the latter describes the proportion of valid data to all data. Both can be output simultaneously.

[0106] Individualized updating of standard movement templates is necessary because a uniform standard movement template is difficult to adapt to trainees with varying functional levels. One approach involves selecting effective movement segments from the trainee's historical training that yield evaluation results better than a preset evaluation level (e.g., scores in the top 20% or higher than the trainee's historical average). The standard movement template is then updated using the skeletal keypoint sequence corresponding to the selected effective movement segments. The update can employ exponential moving average fusion, as shown in equation (12):

[0107] ;

[0108] In equation (12), This is the updated template. This is the current template. The template sequence corresponding to the selected effective action segment (first aligned with the current template's dynamic time regularization) is defined by γ, which is the fusion coefficient and can range from 0.1 to 0.3, with 0.2 being preferred. To prevent the template from drifting as the trainee's ability deteriorates, an upper limit can be set on the cumulative update magnitude of the template (e.g., the cumulative offset of the template at any joint angle does not exceed 10°), or updates can be allowed only when the evaluation results continuously improve.

[0109] The problem addressed by individualized template updates is that standardized movement templates reflect the average level of healthy individuals. Directly using these templates as assessment benchmarks for elderly trainees with cognitive impairment leads to consistently low assessment results, failing to reflect the trainee's true progress and undermining their confidence. By updating the template with the trainee's own historically high-quality and effective movement segments, the assessment benchmark is anchored to their achievable level, thus giving the assessment results an individualized meaning "relative to their own progress." Furthermore, because template updates only select effective movement segments after double-gated filtering, unsuitable segments are not included in the template, ensuring that the template update itself is not contaminated—demonstrating the overall effect of combining double-gated purification with template updates.

[0110] Example 1: Home-based lower limb training for elderly people in the early stages of Alzheimer's disease:

[0111] A 72-year-old elderly person with early-stage Alzheimer's disease underwent sit-stand training at home, with a monocular camera capturing data at 30 frames per second. During a 10-minute training session, the elderly person got up and left the training area three times (a total of 46 seconds), and walked around in the frame twice. Using a control method based on the amplitude of movement, all of the walking (large-amplitude movements) was incorrectly classified as valid data. In this embodiment, the three instances of leaving the training area (missing key points for more than 2 seconds) were removed, and the two walking segments were removed based on the task amnesia type (normal stability but correlation below 0.65 for 5 seconds). The final valid total duration of the action segments was approximately 5.8 minutes, with a confidence level C≈0.58. Compared to the overall assessment results without these exclusions, the correlation coefficient between the assessment results in this embodiment and the blind assessment by the rehabilitation physician increased from 0.61 to 0.83 (exemplary data; actual measurements may vary).

[0112] Example 2: Upper limb training for elderly patients with post-stroke cognitive impairment. A 68-year-old elderly patient with post-stroke cognitive impairment underwent upper limb flexion training. During the training, the elderly patient continuously looked at the screen (stability met gating requirements) but forgot the training task, let their hands hang down, and chatted idly. After 7 seconds, the patient was caught by a task forgetting criterion, the assessment timer was paused, and the standard movement demonstration was replayed. The elderly patient followed the demonstration and resumed training. The double gating was passed again, and the assessment timer was resumed after one template cycle. In this example, hysteresis gating (θ_2=0.65, Δ=0.1) was used. The elderly patient's correlation fluctuated multiple times between 0.6 and 0.7. The hysteresis band avoided three unnecessary breaks in the effective movement segment.

[0113] Example 3: Balance training for elderly people with mild cognitive impairment. A 75-year-old elderly person with mild cognitive impairment underwent static balance training. Four effective movement segments were obtained in this training, with a confidence level of C=0.72. The evaluation results were output after weighting according to Equations (10) and (11). The elderly person's evaluation results steadily improved after three consecutive training sessions. The optimal effective movement segment sequence from the three training sessions was merged and updated with γ=0.2 to update the individualized standard movement template. The updated evaluation benchmark matched the elderly person's current ability, and subsequent evaluation results focused more on the progress relative to their own level.

[0114] Effect Verification: To verify the effect of this application, the following control group can be set up:

[0115] Training videos of the same group of elderly trainees with cognitive impairment were used to output assessment results using three methods: (a) overall assessment without screening, (b) post-screening assessment based on movement amplitude, and (c) post-screening assessment using the dual-gated screening method of this application. The correlation coefficients between each method and the baseline were calculated based on the blind review results of rehabilitation physicians. It was expected that the correlation coefficient of (c) would be significantly higher than that of (a) and (b). Specifically, in scenarios where the elderly were walking or leaving their seats, data with large movements might be misjudged as valid, and its correlation coefficient might even be lower than that of (a), thus verifying that this application solves a technical problem that existing screening methods cannot address in this scenario.

[0116] Specifically, the correlation coefficient can be calculated using Spearman's rank correlation coefficient or Pearson's correlation coefficient; the physician blind review can be conducted by two or more rehabilitation physicians independently scoring the same training video according to a unified movement quality scale, and the average score is taken as the evaluation benchmark. In addition to the correlation coefficient, the invalid data error rate (the proportion of non-cooperation segment time included in the evaluation criteria to the total non-cooperation segment time) can also be calculated for each method: the error rate of method (a) is 100%, the error rate of method (b) is still close to 100% in the elderly wandering scenario, while after the dual gating and scenario-based judgment branching of this application, most of the non-cooperation segments are eliminated, and the error rate is significantly reduced. The above comparison results collectively indicate that the improved consistency between the evaluation results and the physician's judgment comes from the elimination of non-cooperation segments, rather than the difference in the evaluation algorithm itself.

[0117] Furthermore, ablation verification can be performed separately for hysteresis gating and cooperative weighting: with hysteresis gating disabled (using only the second threshold as the unique boundary), the number and average duration of effective action segments are counted to observe the degree of fragmentation caused by correlation jitter; with cooperative weighting disabled (equal weighting of each effective action segment), the sensitivity of the evaluation results to segments near the gating boundary is observed. The above ablation results are used to illustrate the respective technical contributions of hysteresis gating and cooperative weighting.

[0118] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0119] The flowcharts involved in this application are merely illustrative examples and are not intended to require or imply that the steps must be performed in the order shown in the flowcharts. As those skilled in the art will recognize, the steps can be performed in any feasible order, as long as logical dependencies are not violated. Words such as “including,” “comprising,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them.

[0120] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A rehabilitation training assessment method based on posture recognition, characterized in that, include: Collect video sequences of trainees performing rehabilitation training movements; Human pose recognition is performed on the video sequence to obtain the skeletal key point sequence of the trainee; Based on the head key points in the skeletal key point sequence, the stability of the trainee's head posture within the sliding time window is determined, and the correlation between the skeletal key point sequence corresponding to the target joint and the action sequence corresponding to the standard action template within the sliding time window is determined. Continuous time periods in which the stability is greater than a preset first threshold and the correlation is greater than a preset second threshold are determined as effective action segments. The effective movement segments are compared with the standard movement template to obtain the evaluation results of the rehabilitation training movements.

2. The rehabilitation training assessment method based on posture recognition according to claim 1, characterized in that: Determining the stability of the trainee's head posture within the sliding time window includes: Based on the aforementioned head key points, calculate the yaw angle and pitch angle of the trainee's head; Calculate the standard deviation of the yaw angle and the pitch angle within the sliding time window, and determine the stability as the reciprocal of the standard deviation; The method further includes: in response to the angle between the trainee's head orientation and a preset reference direction being greater than a preset angle threshold, determining the corresponding time period as a segment in which the trainee did not cooperate in performing the rehabilitation training action.

3. The rehabilitation training assessment method based on posture recognition according to claim 1, characterized in that: Determining the stability of the trainee's head posture within the sliding time window includes: In response to the confidence level of the head key points in the skeletal key point sequence being lower than a preset confidence threshold, the change in the deviation angle of the trainee's body orientation within the sliding time window is determined based on the trunk key points in the skeletal key point sequence, and the displacement velocity of the trainee's center of mass within the sliding time window is determined based on the pelvic key points in the skeletal key point sequence. The stability is determined based on the change in the deviation angle and the displacement velocity of the center of mass.

4. The rehabilitation training assessment method based on posture recognition according to claim 1, characterized in that: Determining the correlation between the skeletal keypoint sequence corresponding to the target joint within the sliding time window and the action sequence corresponding to the standard action template includes: Calculate the joint angle sequence of the target joint within the sliding time window based on the skeletal key point sequence corresponding to the target joint; Calculate the normalized cross-correlation coefficient between the joint angle sequence and the standard angle sequence corresponding to the standard motion template; The normalized cross-correlation coefficient is determined as the correlation degree.

5. The rehabilitation training assessment method based on posture recognition according to claim 1, characterized in that: The step of determining a continuous time period in which the stability is greater than a preset first threshold and the correlation is greater than a preset second threshold as a valid action segment includes: In response to the stability within the sliding time window being greater than the preset first threshold and the correlation increasing from less than the preset second threshold to greater than the preset third threshold, the sliding time window is marked as the start of the effective action segment; In response to the stability within the sliding time window being greater than the preset first threshold and the correlation decreasing from greater than the preset second threshold to less than the preset fourth threshold, the sliding time window is marked as the end of the effective action segment; Wherein, the preset third threshold is greater than the preset second threshold, and the preset fourth threshold is less than the preset second threshold.

6. The rehabilitation training assessment method based on posture recognition according to claim 1, characterized in that: The method further includes: In response to the loss of the skeletal key points corresponding to the target joint in consecutive image frames, or the movement of the trainee's body area out of the video sequence capture frame for more than a first preset duration, the corresponding segment is determined to be a segment in which the trainee did not cooperate in performing the rehabilitation training action. In addition, in response to the stability being greater than the preset first threshold and the correlation being continuously less than or equal to the preset second threshold for more than a second preset duration, the corresponding continuous time period is determined as a segment in which the trainee does not cooperate in performing the rehabilitation training action.

7. The rehabilitation training assessment method based on posture recognition according to claim 1, characterized in that: The step of defining a continuous time period in which the stability is greater than a preset first threshold and the correlation is greater than a preset second threshold as a valid action segment further includes: Remove valid action segments with a duration shorter than the preset fragment duration, and merge adjacent valid action segments with an interval shorter than the preset interval duration; Furthermore, the evaluation result includes a confidence level, which is the proportion of the total duration of the effective action segment to the total duration of the video sequence.

8. The rehabilitation training assessment method based on posture recognition according to claim 1, characterized in that: The method further includes: If a segment in which the trainee fails to cooperate in performing the rehabilitation training movements continues for more than a third preset duration, the assessment timer is paused and a prompt message guiding the trainee to resume cooperation is output. In addition, in response to the recurrence of consecutive time periods in which the stability is greater than the preset first threshold and the correlation is greater than the preset second threshold, the evaluation timing is resumed.

9. The rehabilitation training assessment method based on posture recognition according to claim 1, characterized in that: The step of comparing the effective movement segment with the standard movement template to obtain the evaluation result of the rehabilitation training movement includes: Based on the stability and correlation within the effective action segment, determine the coordination weight corresponding to the effective action segment; The comparison results of the effective action segments are weighted using the aforementioned matching weights to obtain the evaluation result.

10. The rehabilitation training assessment method based on posture recognition according to claim 1, characterized in that: The method further includes: Select effective movement segments from the trainee's historical training performance that have an evaluation result better than the preset evaluation level. The standard motion template is updated with the skeletal keypoint sequence corresponding to the selected effective motion segment.

Images