Visual skeleton rehabilitation action evaluation and occlusion completion method
Patent Information
- Application Number
- CN202610407668.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-03-31
- Publication Date
- 2026-06-26
AI Technical Summary
Existing technologies for visual skeleton rehabilitation movement assessment suffer from problems such as unstable occlusion detection, lack of temporal continuity in determining the cause of occlusion, inability to distinguish the cause of occlusion, inability to differentiate completion strategies, and poor applicability to complex postures.
By extracting the set of key points and confidence scores of the human skeleton frame by frame, a human mask is generated. The occlusion causes are classified by combining the length difference between adjacent bone segments, the displacement difference between key points, and the confidence scores of key points inside and outside the mask. Under the condition of keeping the credible key points unchanged, completion candidates are generated, and occlusion completion is performed by using posterior consistency test and mirror candidate solutions.
It improves the accuracy and stability of occlusion detection, enhances the usability of rehabilitation movement assessment in complex scenarios, ensures that the completion results conform to human body shape and movement patterns, and reduces the misjudgment rate.
Smart Images

Figure CN122290206A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision technology, specifically to a method for assessing and completing visual skeleton rehabilitation movements and occlusion. Background Technology
[0002] Current technologies, such as the one disclosed in CN114926401A, which describes a method, device, equipment, and storage medium for detecting skeletal point occlusion, still have many shortcomings in terms of methods for visual skeleton rehabilitation movement assessment and occlusion processing. This document mainly focuses on skeletal point occlusion detection rather than the construction of stable skeleton sequences required for rehabilitation movement assessment. Its core idea is to acquire aligned color and depth images and rely on pixel-by-pixel alignment between the depth and color images to support subsequent judgments. This means that the system often requires additional acquisition devices such as depth cameras or dual cameras on the hardware side, as well as calibration and alignment processes, resulting in high deployment costs and limited applicability in low-threshold home rehabilitation scenarios. At the same time, alignment errors will directly lead to occlusion misjudgments.
[0003] This scheme constructs regions based on a set of connected skeleton points. It creates a spherical model using the first and last skeleton points and encloses it with a cylindrical model to form the target region. The radius depends on depth information and varies with the distance between the user and the camera. This geometric envelope is prone to errors under complex poses, resulting in regions that are too large or too small, introducing numerous false inclusions or omissions. This is especially problematic when keypoints are obtained using a 2D skeleton point detection algorithm and are subject to jitter; the target region's error is amplified by jitter, leading to unstable occlusion detection. The occlusion determination logic primarily relies on whether the detected skeleton point is located inside the target region. If it is, the depth value or axis depth value of the target region is compared with the detected point to determine occlusion. This criterion essentially relates the geometric relationship of a single frame to the depth of preceding and following occlusions. The static judgment of the system lacks continuity constraints and anomaly suppression mechanisms in the time dimension. When faced with common rehabilitation movements such as rapid arm raising, knee flexion, or overall body displacement and turning, depth noise and posture estimation drift can cause occlusion conclusions to frequently flip between adjacent frames, resulting in jumps in subsequent movement quality assessment indicators. After obtaining the occlusion detection results, the file tends to remove occluded bone points and then perform human posture estimation based on the depth information of unoccluded bone points. In rehabilitation movement assessment tasks, this will directly cause the joint angle chain to break and the temporal feature to be missing, thus making it impossible to stably calculate joint range of motion, symmetry, and trajectory smoothness indicators. Especially in long-term occlusion scenarios of key points at the distal upper limb or lower limb, indicators may become unusable or forced interpolation may occur as an engineering compromise.
[0004] This scheme bases occlusion judgment on the geometric relationship of whether a point is occluded by a limb region, serving more the purpose of occlusion detection itself and 3D pose reconstruction and display. However, it does not distinguish between different sources of occlusion, such as out-of-frame, self-occlusion, and external object occlusion. Therefore, it cannot select differentiated completion strategies based on the cause of occlusion, and it also lacks posterior consistency testing to use re-emergence information to correct occlusion conclusions. This makes it difficult for the system to self-correct when there is temporary external object occlusion or short-term false detection of key points. At the same time, its target region construction is limited to the connection relationship that can form any part of the human torso and to build a skeletal point set. For the fine joint chains that are more concerned in rehabilitation assessment, such as the wrist and ankle, and multi-joint coordinated movements, it will still be affected by the simplification of region modeling and the differences in human body shape, resulting in insufficient generalization. Summary of the Invention
[0005] The purpose of this invention is to provide a method for assessing and completing visual skeletal rehabilitation movements and occlusions, thereby addressing some of the shortcomings and deficiencies pointed out in the background art.
[0006] The present invention adopts the following technical solution to solve the above-mentioned technical problems: a visual skeleton rehabilitation movement assessment and occlusion completion method, comprising: acquiring a video sequence, extracting a preset set of human skeleton key points and confidence level frame by frame, and generating a human mask; determining whether a key point is credible, doubtful or unreliable based on the confidence level, the difference in length between adjacent bone segments, the difference in displacement of key points and whether the key point is located within the human mask.
[0007] N consecutive positive integer frames are identified as key points that are doubtful or unreliable as occlusion candidates, and occlusion areas are formed according to the skeleton connection relationship. Based on the distance between the occlusion area and the image boundary and the connectivity of the human body mask, the occlusion causes are classified as out-of-frame, self-occlusion, or external object occlusion.
[0008] While keeping the credible key points unchanged, candidates for completion are generated according to their causes, and candidates that do not meet the bone segment length deviation threshold, displacement jump threshold, or are located outside the human body mask are removed. The completed preset set sequence of human skeleton key points is then output.
[0009] Furthermore, the displacement difference and displacement jump threshold of the key points are determined after the overall human motion is compensated; the overall motion compensation is obtained by the relative pose change of the torso key points determined to be credible in the same frame in adjacent frames; the coordinates of each key point are transformed to the compensated body coordinate system to calculate the displacement difference, and the credible, doubtful or uncredible determination is completed based on the difference.
[0010] Furthermore, the occlusion cause classification includes a posterior consistency check; after the initial classification of the occlusion area is completed and the occlusion candidate key points are determined, the re-emergence of the occlusion candidate key points is detected in the subsequent M frames from the determined time; when the re-emergence position is consistent with the completion candidate within a preset error range, the occlusion cause is corrected; where M is a preset positive integer and is set independently of N.
[0011] Furthermore, the generation of completion candidates based on causes includes generating at least one pair of mirror candidate solutions for the same occluded joint; the mirror candidate solutions are determined by at least two adjacent credible key points and bone segment length constraints; the number of times the skeleton connecting line segments formed by the mirror candidate solutions cross the human mask boundary is used for rejection screening, and the candidate solutions with fewer crossing times and satisfying the bone segment length deviation threshold and displacement jump threshold are selected as the final completion results.
[0012] Furthermore, the posterior consistency test includes a recurrence validity determination; when the occlusion candidate key point is determined to be credible in K consecutive frames in the subsequent M frames, and its bone segment length difference and displacement difference do not exceed the corresponding threshold, it is determined to be a valid recurrence; only when it is a valid recurrence, the occlusion cause is corrected based on the consistency result within the preset error range, where K is a preset positive integer.
[0013] Furthermore, when the reappearance location is inconsistent with the completion candidate, an alternative completion candidate is generated and the bone segment length deviation threshold, displacement jump threshold, and human body mask constraint elimination screening are re-executed; if the alternative completion candidate is consistent with the reappearance location within a preset error range, the occlusion cause corresponding to the alternative completion candidate is used as the correction result.
[0014] Furthermore, when generating the mirror candidate solution, a mirror reference plane is first determined. The mirror reference plane is defined by the line connecting at least two adjacent trusted key points of the occluded joint and the body orientation determined by the trusted key points of the torso. At least one pair of mirror candidate solutions is generated on both sides of the mirror reference plane.
[0015] Furthermore, the number of crossings is obtained by sampling along the skeleton connecting line segment at a preset step size and counting the number of times the sampling point switches between the inside and outside states relative to the human body mask; when the number of crossings is greater than the preset threshold, the corresponding mirror candidate solution is rejected.
[0016] Furthermore, the rejection screening includes bending direction consistency verification; determining the bending direction reference based on the occluded joint and adjacent reliable key points within the preset window before occlusion; rejecting a mirror candidate solution when the bending direction is reversed relative to the reference and exceeds a preset angle threshold; if none are rejected, then a candidate solution consistent with the reference is selected as the final completion result.
[0017] Furthermore, the bending direction consistency verification adopts a hysteresis rule; when the mirror candidate solution reverses the bending direction relative to the reference and the reversal angle exceeds the first angle threshold, the candidate solution is rejected; when the reversal angle is between the first angle threshold and the second angle threshold, the verification result of the previous frame is maintained, wherein the first angle threshold is greater than the second angle threshold.
[0018] The beneficial effects of this invention are as follows: By extracting skeleton key points frame by frame from a video sequence and generating corresponding human body masks, the confidence information is fused with the consistency of bone segment length, displacement continuity, and the consistency between key points and human body regions for judgment, thereby classifying key points into three categories: credible, questionable, and unreliable. The stability of state across multiple consecutive frames is used to distinguish occlusion candidates from single-frame noise, and occlusion regions are formed based on skeleton connectivity, elevating occlusion detection from single-point judgment to structured joint region judgment. The causes of occlusion are differentiated based on the distance between the occlusion region and the image boundary, as well as the connectivity of the human body mask, effectively identifying different causes of occlusion such as drawing, self-occlusion, and external object occlusion. This provides a basis for subsequent differentiated completion strategies, improving the usability of rehabilitation movement assessment in complex scenarios.
[0019] During the completion phase, reliable key points are kept unchanged, and displacement differences and abrupt changes are evaluated in the body coordinate system after overall human motion compensation. This reduces misjudgments caused by overall translation or turning and improves the effectiveness of temporal continuity constraints. In terms of candidate generation and selection, mirror candidate solutions are generated for the same occluded joint, and rapid rejection is performed based on the number of times the skeletal connecting line segments cross the human mask boundary. Optimization is then performed by combining bone segment length deviation thresholds and displacement abrupt change thresholds, enabling stable selection of completion results that better conform to human shape and movement patterns even with multiple solutions. A posterior consistency test and recurrence validity judgment are introduced. Occlusion causes are corrected only under the condition of continuous reliability and valid recurrence that meets threshold constraints. In cases of inconsistency, alternative candidates are generated for backtracking correction. Simultaneously, bending direction consistency checks and hysteresis rules suppress critical jitter, significantly improving the accuracy, stability, and repeatability of occlusion cause judgment and key point completion, thereby enhancing the reliability of rehabilitation movement assessment index calculations. Attached Figure Description
[0020] Figure 1 This is a simplified logic flowchart for visual skeleton occlusion detection and completion in this invention.
[0021] Figure 2 This is a schematic diagram illustrating the joint determination of displacement difference and bone segment deviation rate after compensation of key points of the right knee in Embodiment 1 of the present invention.
[0022] Figure 3 This is a schematic diagram of the occlusion candidate triggered by the rule of consecutive N frames in Embodiment 1 of the present invention.
[0023] Figure 4 This is a schematic diagram of the trajectory of the posterior consistency test triggering substitution completion in Embodiment 1 of the present invention.
[0024] Figure 5 This is a schematic diagram of the construction of the mirror reference plane and the generation of elbow point mirror candidates in Embodiment 2 of the present invention.
[0025] Figure 6 This is a schematic diagram of the sampling and rejection screening of the number of crossings by step size in Embodiment 2 of the present invention.
[0026] Figure 7 This is a schematic diagram of the bending direction consistency and hysteresis rule suppressing inter-frame jitter in Embodiment 2 of the present invention. Detailed Implementation
[0027] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.
[0028] Combined with appendix Figure 1 This invention discloses a visual skeleton rehabilitation movement assessment and occlusion completion method. The video sequence can be acquired by a fixed or mobile camera, and the video frame rate and resolution can be set according to the rehabilitation movement assessment requirements. The video sequence is processed frame by frame in chronological order. For each frame, a human pose estimation model is input to obtain a preset set of human skeleton key points and the confidence level of each key point. The key point set includes trunk key points and limb key points. For the same frame, a human segmentation model or foreground segmentation algorithm is input to obtain a human mask. The human mask is used to represent the human pixel region and can further extract human boundary and connected component information.
[0029] An initial judgment is made based on the confidence level of the keypoint and a preset confidence threshold. If the confidence level is lower than the threshold, the keypoint is marked as a doubtful or unreliable candidate. The length difference between adjacent bone segments is calculated. This length difference represents the change in length of the bone segment formed by the keypoint and its connected keypoints in two adjacent frames, reflecting whether there are any abrupt changes in bone segment length. Next, the displacement difference of the keypoint is calculated. This displacement difference represents the change in displacement of the keypoint in two adjacent frames, reflecting whether there are any abrupt changes in the keypoint's motion trajectory.
[0030] In the comprehensive judgment, a keypoint is judged as reliable when its confidence level meets the threshold, the length difference between adjacent bone segments does not exceed the length threshold, the displacement difference of the keypoint does not exceed the displacement threshold, and the keypoint is located within the human body mask. If a keypoint meets some of these conditions but has a slight exceedance of any single condition or exhibits uncertainty, it is judged as questionable. If a keypoint's confidence level is significantly lower than the threshold, the length difference between adjacent bone segments exceeds the length threshold, the displacement difference of the keypoint exceeds the displacement threshold, or the keypoint is located outside the human body mask, meeting a preset inconsistency condition, it is judged as unreliable.
[0031] After completing the frame-by-frame keypoint credibility status labeling, stability is assessed for abnormal states in the time dimension. A state sequence is established for each type of keypoint, and the duration for which the keypoint is judged as questionable or untrustworthy within consecutive frames is statistically analyzed. When the same keypoint is judged as questionable or untrustworthy for a consecutive preset positive integer N frames, it is identified as a candidate occlusion keypoint. N is used to suppress false detections and short-term jitter in a single frame, and N can be set according to the video frame rate and motion speed. The candidate occlusion keypoints are further aggregated according to skeleton connectivity, and connected subsets are constructed using the candidate occlusion keypoint and its connected skeleton segments as basic units to obtain the occlusion region.
[0032] When the minimum distance is less than a preset boundary distance threshold, the occlusion cause is determined to be out of frame, indicating that a part of the human body is outside the field of view, causing keypoint anomalies. The connectivity of the human body mask is analyzed, including the number of connected components and whether there are obvious breaks or splits in the mask near the occlusion area. When the human body mask maintains a single connected component and the occlusion area is located inside the human body mask, and there are no missing features adjacent to the image boundary, the occlusion cause is determined to be self-occlusion, indicating that the human body posture causes a local area to be occluded by other body parts. When the above conditions are not met and the human body mask is segmented into multiple connected components near the occlusion area or has occlusion boundaries inconsistent with the human body shape, the occlusion cause is determined to be external object occlusion, indicating that the human body is partially covered by an external object, causing keypoint extraction failure.
[0033] The completion process is performed within each frame, and the coordinates of key points that have been determined to be reliable remain unchanged during the completion process, using reliable key points as reference benchmarks for geometric and motion constraints. For each occluded region, a corresponding candidate generation rule is selected based on the cause of occlusion, and at least one completion candidate solution is generated. The completion candidate solution includes the estimated coordinates of the occluded candidate key points and the related bone segment geometric relationships determined therefrom.
[0034] The algorithm fits the motion direction and velocity of keypoints using reliable coordinates from several frames prior to the occlusion, and then extrapolates the keypoints along this direction to positions near the image boundary. Simultaneously, it restricts the extrapolation results from crossing non-edge positions within the main body region of the human mask to avoid misclassifying out-of-frame views as occlusion within the mask. When the occlusion is caused by self-occlusion, completion candidates are generated based on skeleton topology constraints and bone segment length constraints. When the occlusion is caused by external objects, completion candidates are generated based on continuity constraints before and after occlusion and human mask morphology constraints.
[0035] Bone segment length deviation is the difference between the bone segment length formed by the occluded keypoint and its connected reliable keypoints in the candidate solution and the reference bone length. Candidates are discarded when the deviation exceeds the bone segment length deviation threshold. Displacement jump is the amount of displacement change between the coordinates of the candidate solution and the coordinates of the corresponding keypoint in the previous frame. Candidates are discarded when the change exceeds the displacement jump threshold. Human mask constraint judgment is performed; candidates are discarded when their coordinates are outside the human mask or located in a non-main connected region of the mask and do not meet the out-of-frame feature. The remaining candidate solutions are sorted in ascending order of the combined cost of bone segment length deviation and displacement jump, and the candidate solution with the lowest cost is selected as the completion result for the occluded keypoints in that frame.
[0036] Overall motion compensation uses trunk-type keypoints as stable references, which include at least a portion of the neck, chest, pelvic center, left and right hips, and left and right shoulders. For each frame, trunk-type keypoints deemed reliable within the frame are preferentially selected to form a trunk reference set, and the corresponding coordinates of the same trunk keypoints are found in adjacent frames. Based on the spatial correspondence between the two frames of trunk reference sets, the relative pose change of the human body is estimated. The relative pose change includes at least two-dimensional translation and in-plane rotation, and may further include a scale factor to adapt to changes in camera distance.
[0037] For each keypoint within the current frame, its pixel coordinates are inversely aligned using a compensation transformation, ensuring that reliable keypoints of the torso type overlap as much as possible in adjacent frames, thus obtaining the compensated keypoint coordinates. Subsequently, the keypoint displacement difference is calculated in the compensated volume coordinate system, where the keypoint displacement difference represents the amount of displacement of the same keypoint between the compensated coordinates of two adjacent frames.
[0038] If the keypoint confidence level meets the requirements, the compensated displacement difference does not exceed the displacement threshold, the bone segment length difference meets the length constraint, and the keypoint is located within the human body mask, it is considered reliable. If a keypoint has a single minor exceedance but does not form a persistent abnormality, it is considered questionable. If the compensated displacement difference exceeds the displacement jump threshold, is significantly inconsistent with the human body mask area, or is accompanied by abnormal abrupt changes in bone segment length, it is considered unreliable.
[0039] After completing the initial classification of occluded areas and identifying candidate keypoints, this identification time is recorded as the start time. Reappearance detection is then performed on these candidate keypoints within a subsequent preset positive integer M frames after the start time. Reappearance detection identifies the time and location at which keypoints recover from an occluded state to an observable state, thus providing a basis for correcting the cause of occlusion. M can be set according to the action cycle and the duration of occlusion, and M and N, used for candidate occlusion determination, are set independently to allow the temporal stability determination and posterior verification window to adapt to different false detection suppression requirements.
[0040] For the same occluded candidate keypoint, in each subsequent frame, it is checked whether the keypoint has been successfully extracted and meets the basic conditions for reliability or doubt. When the keypoint becomes usable again in a certain frame, that frame is recorded as the reappearance frame and its reappearance position is recorded. The completion candidate or final completion result generated for the keypoint during the occlusion period is retained, and the predicted position of the completion candidate in the reappearance frame is compared with the reappearance position. The consistency comparison is judged using a preset error range. When the Euclidean distance or the coordinate difference in the horizontal and vertical directions does not exceed the preset error range, the reappearance position is considered to be consistent with the completion candidate.
[0041] Correction methods may include correcting external object occlusion to self-occlusion, correcting self-occlusion to out-of-view, or switching between out-of-view and external object occlusion. The correction rules are jointly determined by the initial classification features and the consistency results.
[0042] After an occluded candidate keypoint enters the posterior verification window, its extraction result and credibility status are continuously monitored for each of the subsequent preset positive integer M frames. When the keypoint is detected to reappear, it is not immediately considered a recurrence that can be used to correct the cause, but rather it enters the validity determination process.
[0043] If a candidate occluded keypoint is deemed reliable for K consecutive positive integer frames within the subsequent M frames, it is considered to have a stable observation basis. Further calculations are then performed on the bone segment length difference and keypoint displacement difference within these K consecutive frames. The bone segment length difference is calculated for adjacent frames and compared with a bone segment length threshold. The displacement difference is calculated after compensation for overall human motion and compared with a displacement threshold. If the bone segment length difference and displacement difference within these K consecutive frames do not exceed the corresponding length threshold, the reappearance of the keypoint is considered a valid reappearance.
[0044] The reappearance location of a valid reappearance frame is compared with the predicted location of the completion candidate generated during the occlusion period at the same time. When the two are consistent within a preset error range, the correction or confirmation of the occlusion cause is triggered, and the corrected occlusion cause is used to update the completion strategy selection of subsequent frames. If the valid reappearance condition is not met, the occlusion cause is not corrected based on the consistency result, but the initial classification result remains unchanged to reduce erroneous corrections caused by short-term noise.
[0045] If the comparison results show that the reappearance location is inconsistent with the completion candidate, it is considered that there is a deviation between the current completion candidate and its corresponding occlusion cause. Therefore, the alternative completion process is initiated to reinterpret the occlusion event and correct the completion result. Inconsistency can be determined using distance or component difference metrics. When the difference between the reappearance location and the completion candidate in coordinate space exceeds a preset error range, it is considered inconsistent.
[0046] The generation method can include switching the candidate generation rules corresponding to the two types of causes: self-occlusion and external object occlusion, or expanding the search range and introducing a new geometric solution set under the original cause rules, thereby covering the real trajectories that the original completion candidates failed to hit. Subsequently, each alternative completion candidate is re-selected and eliminated. The bone segment length deviation constraint is used to ensure that the bone segment length formed by the alternative candidate and the adjacent credible key points is consistent with the reference bone length. The displacement jump constraint is used to ensure that the change of the alternative candidate is continuous in adjacent frames. The human body mask constraint is used to ensure that the alternative candidate falls within the main body area of the human body mask and is consistent with the human body shape.
[0047] If a candidate replacement for occlusion is consistent with the reappearance location within a preset error range, it is considered to better match the actual observation results after occlusion removal. The occlusion cause corresponding to this candidate replacement is then used as a correction result to update the occlusion cause record for that occluded region and the completion strategy selection for subsequent frames. If multiple candidates replacements meet the consistency condition, the candidate with the lowest combined cost of bone segment length deviation and displacement jump is selected as the final replacement result to ensure the optimality of the completion result in terms of geometric consistency and temporal continuity.
[0048] When there are at least two reliable keypoints in the adjacent bone segments of an occluded joint, geometric constraints are established using these reliable keypoints as known endpoints, and the position of the occluded joint is solved by combining the bone segment length constraints. The bone segment length constraints can be given by a pre-defined human skeleton model, or the reference bone length can be obtained statistically from several frames before the occlusion occurs.
[0049] The mirrored candidate solutions are connected to adjacent trusted key points to form skeleton connecting segments, and the geometric consistency of these segments within the human body mask is assessed. When a skeleton connecting segment significantly extends beyond the human body mask or repeatedly crosses its boundary, it indicates that the candidate solution does not conform to the human body shape and is therefore rejected. The number of crossings can be obtained by sampling the skeleton connecting segments and counting the number of state transitions from inside to outside the mask at the sampling points. A higher number of crossings indicates that the candidate solution is less consistent with the human body region defined by the mask.
[0050] The bone segment length deviation is calculated and compared with a bone segment length deviation threshold; candidate solutions that cause significant abnormalities in bone segment length are discarded. Then, displacement jumps are calculated and compared with a displacement jump threshold; candidate solutions that disrupt temporal continuity are discarded. For candidate solutions that simultaneously satisfy both the bone segment length deviation threshold and the displacement jump threshold, the candidate solution with fewer crossings is preferentially selected as the final completion result. If the number of crossings is the same, the candidate solution with the lower combined cost of bone segment length deviation and displacement jumps is selected as the final completion result.
[0051] A mirror reference plane is used to define the symmetry of the occluded joint positions and is consistent with the current human posture. At least two trusted keypoints adjacent to the occluded joint are selected. These adjacent trusted keypoints can be upstream and downstream joints directly connected to the occluded joint, or two adjacent joints on the same limb chain. The line connecting the adjacent trusted keypoints is used as a local geometric baseline to define the principal orientation and length constraints of the limb containing the occluded joint.
[0052] The transverse direction of the torso is determined by the lines connecting the left and right shoulders and the left and right hips, and the longitudinal direction of the torso is determined by the line connecting the center of the chest and the center of the pelvis, thus obtaining the body orientation vector. This body orientation vector, along with the local geometric baseline, is used to define a mirror reference plane, ensuring that the mirror reference plane reflects the local limb structure while maintaining consistency with the overall torso orientation. The mirror reference plane can be defined as a plane passing through the local geometric baseline whose normal direction is determined by the body orientation vector, or as a symmetrical plane with the local geometric baseline as its axis and whose orientation aligns with the torso orientation.
[0053] The positions of occluded joints are determined under the constraint of bone segment length, and the resulting solutions are projected onto both sides of a mirror reference plane to form pairs of mirror candidate solutions. Each mirror candidate solution forms a skeleton connection relationship with its adjacent reliable keypoints, while maintaining the consistency between bone segment length and joint topology constraints.
[0054] For each mirror candidate solution, the occluded joint is connected to its adjacent trusted keypoints to form a skeleton connection segment. This skeleton connection segment can include one or more segments, each corresponding to an adjacent bone segment on either side of the occluded joint. Sampling is performed along each skeleton connection segment at a preset step size, set in pixels. The step size can be adjusted based on the video resolution and keypoint localization accuracy. For each sampled point, it is checked whether the sampled point falls within the human body mask coverage area, thus obtaining the internal / external state of the sampled point. The internal / external state includes two categories: located inside the human body mask and located outside the human body mask.
[0055] A switch occurs when the state of two adjacent sampling points changes from inside to outside the human body mask, or vice versa. The total number of switches for the same skeleton connecting line segment is accumulated to obtain the traversal count for that segment. If a mirror candidate solution contains multiple skeleton connecting line segments, the traversal counts of each segment are summed to obtain the traversal count for that mirror candidate solution. A higher traversal count indicates that the skeleton connecting line segment repeatedly enters and exits near the boundary of the human body mask, typically corresponding to a candidate solution that is inconsistent with the human body's outline or that the candidate solution falls outside the human body region.
[0056] When the number of crossings exceeds a preset threshold, the corresponding mirror candidate solution is rejected and does not participate in the subsequent screening for bone segment length deviation and displacement jump. When the number of crossings is not greater than the preset threshold, the corresponding mirror candidate solution is retained and proceeds to the next step of consistency constraint screening.
[0057] The bending direction consistency check is performed on joints with a clear flexion-extension direction. Before the occluded joint enters the occlusion state, a preset window frame before occlusion is selected as a reference window. The reference window is located before the occlusion occurs and contains keypoint results from multiple consecutive frames. For each frame within the reference window, the bending geometry of the joint is constructed using the occluded joint and its two adjacent trusted keypoints. The adjacent trusted keypoints can be the upstream joint and the downstream joint. The bending direction of the joint is calculated based on the vector from the upstream joint to the occluded joint and the vector from the occluded joint to the downstream joint. The bending direction statistics of most frames within the reference window are used as the bending direction benchmark to suppress the influence of single-frame noise on the benchmark.
[0058] The comparison method can be based on the directional rotation direction of the joint angle or on the consistency of the normal direction composed of two vector segments, thereby obtaining the reversal angle of the mirror candidate solution relative to the reference. When the mirror candidate solution causes the bending direction to reverse relative to the reference and the reversal angle exceeds a preset angle threshold, the candidate solution is considered inconsistent with the continuity of human motion, and thus the mirror candidate solution is rejected.
[0059] Candidate solutions whose bending direction matches the baseline are prioritized as the final completion result to ensure that the bending direction of the completed joint remains consistent with the motion trend before the occlusion occurred. If multiple candidate solutions match the baseline, they are comprehensively ranked based on the number of crossings, bone segment length deviation, and displacement jump cost, and the candidate solution with the lowest comprehensive cost is selected as the final completion result.
[0060] Hysteresis rules introduce stability constraints on the time series, making the rejection and retention of candidate solutions memory-based, thereby improving the continuity of the completion results. For each frame's mirror candidate solution, the reversal angle of its bending direction relative to the bending direction reference is calculated, and the reversal angle is compared with a first angle threshold and a second angle threshold, where the first angle threshold is greater than the second angle threshold. The first angle threshold is used to trigger a strong rejection, and the second angle threshold is used to define the retention interval.
[0061] When the reversal angle is between the first and second angle thresholds, the system does not immediately switch between rejection and retention based on the current frame's reversal angle; instead, it maintains the verification result from the previous frame. The previous frame's verification result can be represented as a rejection of the same mirror candidate solution or an identifier of the ultimately selected candidate solution, thus suppressing short-term fluctuations. When the reversal angle does not exceed the second angle threshold, the candidate solution is determined to be consistent with or nearly consistent with the benchmark, allowing it to be retained for the next screening step or to participate in the final selection.
[0062] Example 1:
[0063] In the lower limb rehabilitation assessment room of a rehabilitation medicine center, a therapist conducted a knee flexion and extension training assessment for patient Li. The acquisition device was a regular mobile phone camera fixed on a tripod, with a resolution of 1920×1080, a frame rate of 30fps, and a acquisition distance of about 2.8m. The background included parallel handrails and common rehabilitation equipment such as elastic bands. The assessment goal was to output a continuous and stable sequence of skeletal key points and calculate the range of motion and movement rhythm indicators of the knee joint.
[0064] The system consists of a video acquisition module, a key point extraction module, a human body mask segmentation module, a confidence determination and occlusion detection module, an occlusion cause classification module, a completion and posterior consistency verification module, and an evaluation output module. The key point extraction module outputs a preset set of human skeleton key points and corresponding confidence levels, while the human body mask segmentation module outputs a binary human body mask frame by frame to support key point spatial constraints and connectivity analysis.
[0065] Data acquisition and preprocessing include reading video frame by frame and simultaneously outputting key point coordinates and confidence scores. At the same time, a human body mask is generated for each frame. The human body mask is used to determine whether key points are located within the human body area and to classify the cause of occlusion. The key point set includes lower limb points such as knee, hip, and ankle, as well as trunk points such as pelvic center, chest center, and neck center. Subsequent overall motion compensation only uses the three types of trunk key points that are judged to be reliable: pelvic center, chest center, and neck center.
[0066] The confidence thresholds are set as a credible threshold of 0.70 and a doubtful threshold of 0.40. Keypoints with a confidence level greater than or equal to 0.70 are initially judged as credible, those between 0.40 and 0.70 are initially judged as doubtful, and those less than 0.40 are initially judged as unreliable. At the same time, three joint verifications are introduced: the difference in length between adjacent bone segments, the difference in displacement after compensation, and whether the keypoint is located inside the human body mask. If the keypoint is located outside the human body mask, it is directly downgraded to unreliable. If the displacement difference or the deviation in bone segment length exceeds the threshold, the credible level is downgraded to doubtful or unreliable.
[0067] The overall motion compensation is based on the translation of the reliable torso point between two adjacent frames. First, the displacement of the pelvic center in adjacent frames is calculated as the whole body translation compensation. Then, the compensation displacement difference of each key point is calculated in the body coordinate system. The compensation displacement difference is used for displacement difference judgment and displacement jump judgment. The displacement jump threshold is set to 15px, and the compensation displacement difference threshold is set to 10px.
[0068] The displacement difference after compensation is calculated using the following formula, where The coordinates of the key points in the current frame. The coordinates of the key points in the previous frame, Δ The total translational compensation amount estimated from the reliable torso point. To compensate for the displacement difference:
[0069]
[0070] From frame 120 to frame 121, the pelvic center moves from (520, 620) to (528, 618), resulting in Δ. =(8,-2), the right knee keypoint is at frame 120. =(610,780), detected in frame 121. =(628,770), then:
[0071]
[0072] This leads to the appendix Figure 2 The calculation results shown 12.81px, which is greater than the compensated displacement difference threshold of 10px but less than the displacement jump threshold of 15px, indicates that the motion consistency is abnormal but has not reached a jump; combined with the attached... Figure 2 Based on the mid-bone segment deviation rate information and key point confidence information, the right knee in this frame is downgraded from credible to questionable and enters the subsequent occlusion candidate judgment process.
[0073] The length deviation between adjacent bone segments is calculated using the following deviation rate method, where The length of the bone segment in the current frame. The reference length is the median length within the 30 frames prior to the occlusion, and the deviation rate threshold is set to 12%.
[0074]
[0075] The length of the right femur segment is taken as the Euclidean distance from the hip to the knee, the reference length before occlusion. px, obtained in frame 121 due to keypoint drift ,but:
[0076]
[0077] This leads to the appendix Figure 2 The calculation results shown =15%, since it is greater than the 12% threshold, the length constraint further supports the judgment of the key point as untrustworthy, thereby avoiding erroneous skeleton stretching into the completion output.
[0078] Occlusion candidate determination adopts the continuous frame rule, setting And with the posterior window Independent settings are implemented. When the same keypoint is judged as questionable or unreliable for three consecutive frames, it is marked as an occlusion candidate. Occlusion candidates are aggregated according to skeleton connection relationships to form occlusion regions. When an occlusion region includes adjacent points of the right knee and right ankle, it is considered the same region to uniformly determine the cause of occlusion; Appendix Figure 3 The judgment level of the right knee keypoint in frames 119 to 123 is expressed using a heatmap matrix, and the continuous counting trigger position is marked with a symbol to visualize the continuous count. Frame rules.
[0079] The occlusion cause classification uses both the distance between the occluded area and the image boundary and the connectivity of the human mask. The boundary distance threshold is set to 25px. When the minimum distance from any candidate key point in the occluded area to the image boundary is less than 25px, it is judged as out of frame. When the minimum distance is greater than or equal to 25px, the connectivity of the human mask is further judged. If the human mask has obvious breaks and the break position is spatially adjacent to the occluded area, it is judged as external object occlusion. If the mask remains connected and the candidate key point falls inside the mask but the confidence and motion consistency are abnormal, it is judged as self-occlusion.
[0080] Two types of occlusion occurred during this knee flexion and extension training. The first was that the therapist adjusted the elastic band on the patient's right side, causing a local break in the mask near the right knee and obscuring key points. The system initially judged this as external object occlusion. The second was that when the patient's knee flexion range was large, the lower leg and thigh overlapped, resulting in a decrease in the confidence of the knee point and abnormal displacement consistency. The system judged this as self-occlusion. Both of these triggered occlusion candidates and entered the completion process.
[0081] Candidate completion generation is performed while keeping the credible keypoints unchanged. In the case of external object occlusion, the initial position of the candidate is obtained by extrapolating the velocity of the keypoint before the occlusion occurs and combining it with the bone segment length reference constraint. In the case of self-occlusion, the candidate is inferred from the geometric relationship of the bone segment formed by adjacent credible points. Then, a three-fold elimination screening is uniformly performed. If the candidate causes the bone segment deviation rate to be greater than 12%, it is eliminated. If the candidate causes the displacement difference after compensation to be greater than 15px, it is eliminated. If the candidate falls outside the human body mask, it is eliminated. Finally, the completed keypoints that pass the screening are output to form a continuous skeleton sequence.
[0082] Posterior consistency testing is used to correct occlusion causes and suppress false completions. It sets... frame, The frame has an error range threshold of 6px. Starting from the moment an occlusion candidate is determined, the keypoint is checked for reappearance within the next 8 frames. The validity of reappearance is determined by two consecutive frames being deemed reliable, the corresponding bone segment deviation rate not exceeding 12%, and the compensated displacement difference not exceeding 10px. Only after valid reappearance is a consistency comparison performed. If the reappearance location is consistent with the current completion candidate within 6px, the completion is maintained and the occlusion cause is confirmed. If they are inconsistent, alternative completion candidate generation is triggered. (Appendix) Figure 4 Frames 121 to 123 are represented by a striped background of occlusion areas. The contrast between extrapolation completion and substitution completion is illustrated using both trajectories and scatter points, while a consistency threshold band is used to indicate the effect. scope.
[0083] The replacement candidate generation adopts a reappearance position reverse correction strategy. That is, in the reappearance frame, the reappearance coordinates are used as anchor points to push back to the middle frame of the occlusion interval and perform smooth interpolation combined with bone segment length reference. Then, the bone segment deviation rate, displacement jump, and mask constraint elimination screening are re-executed. If the replacement candidate and the reappearance position are consistent within 6px, the occlusion cause corresponding to the replacement candidate is used as the correction result, thereby correcting the case where initially judged as self-occlusion but actually caused by external object occlusion to external object occlusion, or vice versa. Figure 4 In the corresponding calculation results, taking frame 124 as an example, the consistency error is: ,satisfy By passing the consistency check, the completion result can be maintained without triggering alternative completion, and the appended... Figure 4 Alternative completion trajectories are also provided as comparative examples to illustrate how trajectory abrupt changes in occluded regions can be further reduced when needed.
[0084] Table 1 below provides a small sample of frame-level data for right knee keypoints near the occlusion point. The bone segment deviation rate in the table is the ratio of the bone segment from the right hip to the right knee. The judgment result is based on a comprehensive assessment of confidence level, inside and outside the mask, and It is concluded that the occlusion candidates are continuous Once the frame rules are determined, the selection criteria are used to determine whether the candidates for completion meet the triple elimination conditions.
[0085]
[0086] In terms of performance comparison, the key points of the right knee were manually annotated and evaluated on the same video. Without overall motion compensation, the displacement difference before compensation was amplified by overall translation, resulting in more misjudgments. The average pixel error (MAE) of the key points increased from 8.6px to 12.4px, and the number of occlusion candidate triggers increased by about 1.5 times. After overall motion compensation, the MAE decreased to 8.6px and the number of false triggers decreased, thereby improving the completion stability.
[0087] Without enabling posterior consistency testing, the reappearance position after the occlusion ends deviates from the extrapolation completion, resulting in an occlusion cause classification accuracy of 82%. Enabling posterior consistency testing and triggering alternative completion when inconsistencies occur improves the occlusion cause classification accuracy to 92%, while reducing the keypoint loss rate from 6.8% to 3.1%, maintaining the continuity of the knee joint range of motion curve within the occlusion range, and reducing the inter-frame fluctuation of the motion rhythm index by approximately 30%.
[0088] Example 2:
[0089] This embodiment was applied to the upper limb training assessment area of a rehabilitation center. Patient Zhang underwent combined shoulder abduction and elbow flexion-extension training. The acquisition device was a regular mobile phone camera fixed to the side of the treatment bed, with a frame rate of 30fps. During the training, the patient held an elastic band in his left hand and occasionally gripped a parallel handrail, which caused external objects to obstruct the elbow and wrist joints at the end of abduction and self-obstruction during flexion-extension. The assessment goal was to output a continuous and stable sequence of key points of the elbow joint even when obstructed, and to calculate the stability and jitter amplitude of the elbow flexion-extension angle curve based on this.
[0090] The system consists of a key point extraction module, a human body mask segmentation module, a mirror reference plane construction module, a mirror candidate generation module, a crossing number rejection filtering module, a bending direction consistency and hysteresis verification module, and a result output module. The key point extraction module outputs shoulder, elbow, and wrist key points and confidence scores frame by frame, while the human body mask segmentation module outputs a binary human body mask for inside and outside discrimination and crossing number statistics.
[0091] The mirror reference plane is jointly defined by the line connecting at least two adjacent reliable keypoints of the occluded joint and the body orientation determined by the reliable keypoints of the torso. The left shoulder and left wrist are selected as adjacent reliable keypoints, and the center of the chest and the center of the neck are selected as reliable keypoints of the torso to determine the body orientation. First, the vector from the center of the chest to the center of the neck is used as the upper body orientation basis vector. Then, the vector from the left shoulder to the left wrist is used as the upper limb axis basis vector. The plane spanned by these two vectors is the mirror reference plane, and the normal direction of this plane distinguishes the two sides of the plane, thereby generating at least one pair of mirror candidate solutions on both sides of the reference plane. Figure 5 The geometric premises of the candidate generation are visualized by using a strip-shaped region to represent the projection of the mirror reference plane and a vector field to represent the body orientation field.
[0092] Mirror candidate generation is performed while keeping the key points already determined to be credible unchanged. For the same occluded elbow point, mirror candidate A and mirror candidate B are generated on both sides of the reference plane. The candidate positions are determined by the geometric constraints provided by the left shoulder point and the left wrist point, as well as the bone segment length constraints. The bone segment length constraints follow the settings of Example 1, with a bone segment deviation rate threshold of 12% and a displacement jump threshold of 15px. If a candidate causes a bone segment length deviation rate in the upper arm or forearm to exceed 12%, the candidate enters the rejection queue. Figure 5 The diamond markers in the diagram show the symmetrical distribution of mirror candidate elbow points A and B on both sides of the reference plane, thus reflecting the generation method of candidate pairs.
[0093] The traversal count rejection filter is used to exclude mirror candidates that are inconsistent with the human body mask. Sampling is performed along the line segment connecting the elbow point and adjacent key points at a preset step size, and the state of the sampling point inside or outside the human body mask is determined. The sampling step size is set. px, threshold for the number of crossings When the number of crossings is greater than When a candidate is rejected, the number of traversals is defined as the sum of the number of state transitions inside and outside adjacent sampling points; where, Figure 6 The sampling state changes of candidate A and candidate B are simultaneously illustrated using state bands and cumulative switching step curves, and marked with threshold lines. Used to support a veto conclusion.
[0094] The number of crossings is calculated using the following expression, where Indicates the first Whether each sampling point is located within the human body mask is a binary state. For indicator functions, Number of times traveled:
[0095]
[0096] If the sampling sequence for candidate A is inner, inner, inner, outer, outer, inner, inner, then the state transition occurs twice: from inner to outer and from outer to inner. Therefore:
[0097]
[0098] because Not greater than =2, candidate A is eliminated by the number of traversals, while the sampling sequence of candidate B is inner, outer, inner, outer, inner, outer, inner, with 6 state transitions, resulting in ,because Candidate B is rejected, thus eliminating mirror solutions with a high number of traversals and a greater ability to cross the human body boundary. This rejection relationship is... Figure 6 The phenomenon of the cumulative switching step curve crossing the threshold line is intuitively demonstrated.
[0099] The bending direction consistency check is used to further ensure that the elbow bending direction is consistent with the trend before occlusion, even if the number of crossings has not been rejected. A reference window length for the bending direction is set. The reference direction is obtained by statistically analyzing the bending direction formed by the elbow point relative to the shoulder and wrist points within the 5 frames prior to occlusion. For each mirror candidate, the angle of reversal between its bending direction and the reference direction is calculated. When the angle is reversed and exceeds the threshold, the operation is rejected; among which, Figure 7 The threshold bands are used to illustrate the pass zone, hysteresis zone, and rejection zone, and the time series and decision bands are used to represent the suppression effect of the hysteresis rule on inter-frame jitter.
[0100] The reversal angle is calculated using the cosine of the angle between the vectors, where... The reference bending direction vector, Candidate bending direction vector:
[0101]
[0102] Pick Candidate A corresponds to ,but:
[0103]
[0104] Candidate B corresponds ,but:
[0105]
[0106] First angle threshold Second angle threshold and ,when When a candidate is rejected, The previous frame's verification result is retained to create hysteresis, when It is assumed that the directions are consistent and pass the verification, therefore Pass directly, and Exceed It was rejected.
[0107] Hysteresis rules are used to reduce inter-frame jitter near a threshold. During the occlusion edge stage, if the inversion angle of a candidate frame falls within... and In between, the judgment is not immediately flipped but the result of the previous frame is maintained, so that the elbow flexion and extension angle curve does not produce high-frequency jitter due to the switching between the candidate and the mirror solution on both sides. In particular, when the elastic band is temporarily blocked and then blocked again, the time consistency of the completion result can be maintained. Figure 7 The hysteresis decision band in the image is expressed by padding to show the hold behavior, thus visually indicating that frequent flipping does not occur near the threshold.
[0108] The final candidate selection adopts a step-by-step rejection strategy. First, the bone segment deviation rate and displacement jump constraint are used to eliminate candidates. Then, the number of crossings is used to reject candidates. The consistency of bending direction and hysteresis are checked. When only one candidate remains, the candidate is directly output as the final completion. When multiple candidates remain and none of them are rejected, the candidate that is consistent with the bending direction benchmark and has fewer crossings is selected as the final result.
[0109] The experimental data is presented using a small number of frames near the occlusion point, with a threshold set for the number of traversals. =2, and calculate the number of crossings and the reversal angle for candidate A and candidate B respectively, while recording whether hysteresis is triggered and whether it is finally selected. The hysteresis retention is shown in Table 2 below. The conclusion from the previous frame is used.
[0110]
[0111] In terms of performance comparison, without filtering by the number of crossings, mirror candidate B is more likely to be mistakenly selected when obscured by external objects, resulting in an error completion rate of approximately 9%. Enabling the filtering by the number of crossings and setting... After the error completion rate dropped to 3%, the elbow angle curve experienced inter-frame jumps near the threshold when no hysteresis rule was applied, and the variance of the angle curve increased to about 1.4 times the original value. After enabling the hysteresis rule, the inter-frame jitter amplitude decreased by about 30%, thus making the evaluation of elbow flexion and extension rhythm more stable.
[0112] By generating at least one pair of mirror candidate solutions on both sides of the occluded joint using a mirror reference plane, and combining the rejection of crossing times and the consistency check of bending direction, and introducing hysteresis rules to stabilize inter-frame determination, continuous completion output of key points of the elbow joint is achieved in upper limb training scenarios with self-occlusion and external object occlusion, and reliable data support is provided for rehabilitation movement angle curves and shaking indicators.
Claims
1. A method for assessing and completing visual skeletal rehabilitation movements and occlusion, characterized in that... include: The video sequence is acquired, and the set of key points and confidence scores of the human skeleton are extracted frame by frame, and a human mask is generated. Based on confidence level, difference in length between adjacent bone segments, difference in displacement of key points, and whether the key points are located within the human body mask, key points are determined to be credible, questionable, or unreliable. A series of consecutive positive integer N frames are identified as key points that are questionable or unreliable, and occlusion regions are formed according to the skeleton connection relationship. Based on the distance between the occluded area and the image boundary and the connectivity of the human body mask, the causes of occlusion are classified as out-of-view, self-occlusion, or external object occlusion. While keeping the credible key points unchanged, candidates for completion are generated according to their causes, and candidates that do not meet the bone segment length deviation threshold, displacement jump threshold, or are located outside the human body mask are removed. The completed preset set sequence of human skeleton key points is then output.
2. The visual skeleton rehabilitation movement assessment and occlusion completion method according to claim 1, characterized in that... The displacement difference and displacement jump threshold of the key points are determined after the overall human motion is compensated; the overall motion compensation is obtained by the relative pose change of the torso key points determined to be credible in the same frame in adjacent frames. The coordinates of each key point are transformed into the compensated body coordinate system to calculate the displacement difference, and the reliability, doubt or unreliability is determined based on the difference.
3. The visual skeleton rehabilitation movement assessment and occlusion completion method according to claim 1, characterized in that... The occlusion cause classification includes a posterior consistency test; after the initial classification of the occlusion area is completed and the occlusion candidate key points are determined, the re-emergence of the occlusion candidate key points is detected in the subsequent M frames from the determined time; when the re-emergence position is consistent with the completion candidate within a preset error range, the occlusion cause is corrected. M is a preset positive integer, and it is set independently of N.
4. The visual skeleton rehabilitation movement assessment and occlusion completion method according to claim 1, characterized in that... The generation of completion candidates based on causes includes generating at least one pair of mirror candidate solutions for the same occluded joint; the mirror candidate solutions are determined by at least two adjacent credible key points and bone segment length constraints; the number of times the skeleton connecting line segments formed by the mirror candidate solutions cross the human mask boundary is used for rejection and screening, and the candidate solutions with fewer crossing times and satisfying the bone segment length deviation threshold and displacement jump threshold are selected as the final completion results.
5. The visual skeleton rehabilitation movement assessment and occlusion completion method according to claim 3, characterized in that... The posterior consistency test includes a recurrence validity determination. When the occlusion candidate key point is determined to be credible in K consecutive frames in the subsequent M frames, and the difference in bone segment length and displacement does not exceed the corresponding threshold, it is determined to be a valid recurrence; only when it is a valid recurrence, the cause of occlusion is corrected based on the consistency result within the preset error range, where K is a preset positive integer.
6. The visual skeleton rehabilitation movement assessment and occlusion completion method according to claim 3, characterized in that... When the reappearance location is inconsistent with the completion candidate, an alternative completion candidate is generated and the bone segment length deviation threshold, displacement jump threshold and human body mask constraint elimination screening are re-executed; If the alternative completion candidate and the reappearance position are consistent within a preset error range, then the occlusion cause corresponding to the alternative completion candidate is used as the correction result.
7. The visual skeleton rehabilitation movement assessment and occlusion completion method according to claim 4, characterized in that... When generating the mirror candidate solution, a mirror reference plane is first determined. The mirror reference plane is defined by the line connecting at least two adjacent trusted key points of the occluded joint and the body orientation determined by the trusted key points of the torso. At least one pair of mirror candidate solutions is generated on both sides of the mirror reference plane.
8. The visual skeleton rehabilitation movement assessment and occlusion completion method according to claim 4, characterized in that... The number of crossings is obtained by sampling along the skeleton connecting line segment at a preset step size and counting the number of times the sampling point switches between the inside and outside states relative to the human mask; when the number of crossings is greater than the preset threshold, the corresponding mirror candidate solution is rejected.
9. The visual skeleton rehabilitation movement assessment and occlusion completion method according to claim 4, characterized in that... The rejection screening includes bending direction consistency verification; determining the bending direction reference based on the occluded joint and adjacent reliable key points in the preset window before occlusion; and rejecting the mirror candidate solution when the bending direction is reversed relative to the reference and exceeds the preset angle threshold. If none of the above are rejected, the candidate solution that is consistent with the benchmark is selected as the final completion result.
10. The visual skeleton rehabilitation movement assessment and occlusion completion method according to claim 9, characterized in that... The bending direction consistency verification adopts a hysteresis rule; when the mirror candidate solution reverses the bending direction relative to the reference and the reversal angle exceeds the first angle threshold, the candidate solution is rejected; when the reversal angle is between the first angle threshold and the second angle threshold, the verification result of the previous frame is maintained, wherein the first angle threshold is greater than the second angle threshold.
Citation Information
Patent Citations
Bone point shielding detection method and device, equipment and storage medium
CN114926401A
