A pose recognition method based on multi-view coordinate fusion

Abstract

The application discloses a posture recognition method based on multi-view coordinate fusion, comprising the following steps: performing posture tracking prediction on multi-view image data to obtain multi-view key point data; transforming normalized coordinates of each key point under each view to the same reference coordinate system, fusing according to visibility, supplementing in time sequence, and performing fuse protection to obtain fused key point data; constructing posture feature data from coordinates of the frame of the key point which is not marked as invisible in the fused key point data, inputting the posture feature data into a posture classification model to obtain posture category data; and performing reminding trigger judgment and weighted statistics on the posture category data to obtain training guidance data. The application also discloses corresponding systems and computer readable storage media, so that posture judgment is not interrupted due to single-view occlusion, and coordinate jumping is not caused by view switching.

Description

Technical Field

[0001] This invention relates to the field of posture recognition and data processing technology, and in particular to a posture recognition method based on multi-view coordinate fusion. Background Technology

[0002] Existing posture recognition and alert solutions mostly rely on monocular cameras or wearable sensors. Wearable sensors require continuous wear by the user, which is uncomfortable and easily forgotten. The monocular camera solution has the following drawbacks: First, in single-view observation, key points become directly ineffective when the torso is obstructed by arms or books, leading to interrupted or misjudged posture determination. Even when the user leaves their seat, the system may still output incorrect judgments based on unreliable observations. Second, changes in lighting and background interference in a single view cause the visibility estimation of key points to drift, making the quality of coordinate prediction uncontrollable. Third, if multiple views are added and switched according to a fixed priority, small observation differences between views are amplified into coordinate jumps after switching, and the observation errors from obstructed or blurred views are mixed into the fused coordinates according to weights, causing contamination. Fourth, the jitter of posture categories near the classification boundary is converted into repeated start and stop of the alert signal, forming a noisy alert. Fifth, fixed-interval or unconstrained frequency reduction alerts cannot balance the effectiveness of correction and alert fatigue, and users' compliance with alerts decreases after long-term use. Sixth, obstruction periods are included in the statistics without distinction, causing long-term posture statistics and training guidance to become distorted. Seventh, slow disturbances such as collisions with support structures and temperature deformation cause calibration drift, and the system's judgment accuracy continues to degrade after long-term operation without being detectable. Therefore, there is a need for a pose recognition method that can obtain continuous, reliable key point coordinates without coordinate jumps due to viewpoint switching under conditions of quality fluctuation and occlusion observed from multiple perspectives, and that both alert triggering and long-term statistics are based on reliable observation frames. Summary of the Invention

[0003] This invention proposes a pose recognition method based on multi-view coordinate fusion, comprising: S1. Perform pose tracking and prediction on multi-view image data to obtain multi-view keypoint data; S2. Transform the normalized coordinates of each key point in the multi-view key point data to the same reference coordinate system to obtain the reference coordinates of each key point in each view. When the visibility of a key point in at least one view is not lower than the first visibility threshold, fuse its reference coordinates in each view according to visibility to obtain the fused coordinates of the key point. When the visibility of a key point in each view is lower than the first visibility threshold, use its historical fused coordinates as the coordinates of the current frame within a preset number of supplementary frames. If the preset number of supplementary frames is exceeded, mark it as invisible and exit the posture determination. When the proportion of the number of key points in the temporal supplementation state exceeds a preset proportion threshold, determine that the determination result of the current frame is invalid and obtain the fused key point data. S3. Construct pose feature data from the frame coordinates of key points that are not marked as invisible in the fused key point data, and input the pose classification model to obtain pose category data. S4. Perform reminder trigger determination and weighted statistics on the posture category data to obtain training guidance data.

[0004] Furthermore, the pose tracking prediction includes: performing illumination normalization preprocessing and person detection on each frame of multi-view image data; waiting when no person is detected; locating the region of interest containing the person when a person is detected; and predicting the normalized coordinates and visibility of each key point frame by frame from the image data within the region of interest to obtain multi-view key point data.

[0005] Furthermore, when the normalized coordinates are transformed to the same reference coordinate system, they are first multiplied by the width and height of the image at the viewpoint to restore the pixel coordinates, and then distortion correction is performed on the pixel coordinates. The distortion-corrected pixel coordinates are then inversely projected to the camera coordinate system. After determining the depth of the key points in the camera coordinate system, the coordinates in the camera coordinate system are transformed to the reference coordinate system through a rotation matrix and a translation vector to obtain the reference coordinates of each key point at each viewpoint.

[0006] Furthermore, determining the depth of the key point in the camera coordinate system includes: when the baseline length of the two viewpoints is not less than a preset ratio of the shooting distance, triangulation is performed to recover the depth using the pixel coordinates of the same key point under different viewpoints as corresponding points; when the baseline length is less than the preset ratio or the uncertainty of the triangulated depth exceeds a preset ratio of the depth value, the intersection of the inverse projection ray and the preset constraint plane is taken as the depth, and the preset constraint plane is determined by edge fitting of the preset calibration area during the calibration stage.

[0007] Furthermore, when the determination result of the current frame is deemed invalid, the system enters a circuit breaker state, suspends timing supplementation, and freezes the count update of the number of frames that have been continuously in the timing supplementation state for each key point. During the circuit breaker period, the proportion of key points in the timing supplementation state is continuously monitored. When the proportion remains below the preset proportion threshold within a preset number of consecutive recovery frames, the system exits the circuit breaker state, resumes timing supplementation, and resets the number of frames that have been continuously in the timing supplementation state to zero before restarting the count from the recovery frame.

[0008] Furthermore, the constructed posture feature data includes: features constructed from the current frame coordinates and coordinate ratios of key points in the head, shoulders, elbows, wrists and upper spine that are not marked as invisible. Each feature is normalized by the shoulder distance to make the posture feature data independent of body shape. Among them, the current frame coordinates are fused coordinates in the effective observation frames and historical fused coordinates with decreasing weights in the time-series supplementary frames.

[0009] Furthermore, the reminder trigger determination includes a trigger condition and a stop condition. The trigger condition is that when the posture category data belongs to a preset abnormal posture category and the number of consecutive frames in a single instance reaches a first continuous frame threshold, reminder data corresponding to the preset abnormal posture category is output. The stop condition is that during the reminder data output period, when the number of consecutive frames in a single instance where the posture category data no longer belongs to the preset abnormal posture category reaches a second continuous frame threshold greater than the first continuous frame threshold, the reminder data output is stopped, creating a hysteresis between triggering and stopping. The number of consecutive frames in a single instance targeted by the trigger condition and the stop condition is reset to zero and recounted when the posture category deviates from the currently targeted preset abnormal posture category.

[0010] Furthermore, the minimum interval between two adjacent output reminder data is adaptively adjusted based on historical statistics. The historical statistics are the proportion of the same preset abnormal posture category in the previous statistical period. The minimum interval is determined by a monotonic mapping function. The larger the proportion, the longer the minimum interval. There is no cold start phase without this proportion. The minimum interval is taken as the lower limit of the monotonic mapping function. The parameters of the monotonic mapping function are constrained by the reminder correction effectiveness retention rate, so that the number of effective reminders per unit time after adjustment is not less than the preset proportion before adjustment.

[0011] Furthermore, the weighted statistics include: frames with invalid judgment results are not included in the statistics; frames with a proportion of key points in the time-series supplementation state that exceeds a preset weight reduction threshold but does not exceed the preset proportion threshold are included in the statistics with a weight lower than that of normal frames; and the remaining frames are included in the statistics with full weight. The weighted statistics are used for habit determination.

[0012] Furthermore, after obtaining the reference coordinates of each key point under each viewpoint, and before constructing the pose feature data from the frame coordinates of key points not marked as invisible, online drift verification is performed on the reference coordinates of each key point under each viewpoint. This verification is performed in parallel with subsequent steps. The pose time period is divided with the category switching event of the pose category data as the boundary. For key points corresponding to static body parts, in frames where the visibility of the key point under each viewpoint is higher than the second visibility threshold, the position deviation vector between the reference coordinates of the key point under different viewpoints is calculated. The position deviation vectors are statistically analyzed for the two pose time periods before and after, and the difference is used to obtain the residual deviation that eliminates the real body displacement. When the statistical quantity of the residual deviation on the sliding window exceeds the preset deviation threshold, it is determined that the transformation to the same reference coordinate system has drifted, the pose determination is paused and a recalibration prompt is output. After recalibration, the pose determination is resumed.

[0013] The present invention achieves the following technical effects through the above technical solution: The reference coordinates of all viewpoints at the same key point are transformed to the same reference coordinate system, ensuring comparability. Dual-branch fusion allows the fused coordinates to change continuously with visibility, preventing attitude determination from being interrupted by single-viewpoint occlusion and eliminating coordinate jumps during viewpoint switching. Effective observations from other viewpoints are maintained during single-viewpoint occlusion. Temporal supplementation, point-level exit, and frame-level circuit breaking constitute two levels of protection. Supplementation weights decrease linearly with occlusion, keypoint exit does not produce abrupt feature changes, and any determination results are rejected when a large number of keypoints fail simultaneously. Abnormal attitude determination outputs are only generated from frames with reliable observations. During circuit breaking, the count is frozen, updated, and reset to zero upon recovery, restarting the calculation. After recovery, each keypoint retains a complete supplementation budget. The system produces no erroneous determinations across the entire occlusion spectrum, from short-term occlusion to long-term departure, and exhibits no oscillations after recovery. Coordinate transformations are standardized using distortion correction and extrinsic parameter transformation based on the same calibration benchmark. Depth is selected between triangulation and planar constraints based on baseline conditions. Depth remains determinable even with small baseline arrangements, providing a correct geometric premise for visibility as a fusion weight. All posture feature data are normalized using shoulder distance, making them independent of user body shape. Temporal supplementary frames apply decreasing weights to ensure smooth transitions in feature values. Occlusion and normal observation states share the same classification decision boundary, ensuring consistent accuracy of the classification model in occluded scenarios as in normal observation scenarios. Alert triggering employs a trigger-more-strict-than-stop hysteresis and zero-recalculation mechanism, eliminating boundary jitter from repeated start-stop alerts. Each alert corresponds to a confirmed abnormal posture. The minimum interval is adaptively adjusted using a monotonic mapping function based on historical proportions, constrained by the alert correction effectiveness retention rate. Frequency reduction does not slip into laissez-faire, resulting in overall correction performance superior to either fixed-interval alerts or unconstrained frequency reduction. Weighted statistics and circuit breaker determination share the same proportion index, with occlusion levels continuously graded by the same index at both the determination and statistical ends. Habitual identification data and training guidance data are unaffected by occlusion periods. Online drift verification uses posture time-phased differences to eliminate actual body displacement, intercepting drift before it accumulates into systematic misjudgments. The system maintains initial deployment accuracy without requiring periodic manual shutdowns for calibration. The system automatically assigns tasks to instant reminders and long-term training guidance based on statistical results. Users receive timely reminders in the early stages of correction, and the frequency of reminders automatically evolves with the correction process after the habit is formed, without the need for manual configuration. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the pose recognition process based on multi-view coordinate fusion proposed in this invention. Detailed Implementation

[0015] refer to Figure 1 This invention proposes a pose recognition method based on multi-view coordinate fusion, comprising: S1. Perform pose tracking and prediction on multi-view image data to obtain multi-view key point data.

[0016] The multi-view image data consists of a collection of image frames read by each camera. Pose tracking prediction is a data processing procedure that sequentially performs preprocessing, person detection, and keypoint prediction on each frame of image data. After system initialization, the host computer runs the image acquisition program library to call each camera to read image data. Illumination normalization preprocessing is a process to suppress the impact of changes in indoor lighting on subsequent visibility estimation. The region of interest is the image region where the person is located, circled by the detector from a whole frame containing a large amount of content unrelated to pose recognition, such as the desk, background, and indoor environment. The pose landmark model is a model that predicts keypoints frame by frame and outputs normalized coordinates and visibility. The pose landmark model outputs 33 keypoints for each viewpoint. The normalized coordinates of each keypoint contain three components: horizontal, vertical, and depth. The visibility value ranges from 0 to 1 and is obtained by mapping the output layer of the pose landmark model through the logistic function, representing the probability that the keypoint is located within the human body contour and is not occluded under that viewpoint.

[0017] Specifically, for each frame, histogram equalization and illumination normalization preprocessing are first performed. Then, it is determined whether there is a person in the scene. If no person is detected, the process waits. Once a person is detected, the detector locates the region of interest (ROI). The tracker uses cropped frames of this region as input to predict pose landmarks frame by frame, obtaining multi-view keypoint data. Histogram equalization stabilizes the inter-frame brightness distribution under varying indoor lighting conditions, ensuring that visibility estimation does not drift with lighting. The ROI excludes irrelevant content such as desks, backgrounds, and the indoor environment from the tracker input, significantly reducing the input image area for keypoint prediction. This improves the prediction accuracy and frame rate simultaneously, and background objects no longer participate in keypoint prediction, reducing the false detection rate.

[0018] Furthermore, the pose tracking prediction includes: performing illumination normalization preprocessing and person detection on each frame of multi-view image data; waiting when no person is detected; locating the region of interest containing the person when a person is detected; and predicting the normalized coordinates and visibility of each key point frame by frame from the image data within the region of interest to obtain multi-view key point data.

[0019] Among them, person detection is the process of determining whether there is a person in the frame; waiting when no person is detected means that the frame does not enter the key point prediction stage, and the system continues to collect and detect; frame-by-frame prediction from image data in the region of interest means that the input of the tracker is a cropped frame rather than a whole frame.

[0020] Specifically, illumination normalization preprocessing is performed first, ensuring that both person detection and keypoint prediction are performed on images with stable brightness distribution. After a person is detected, the prediction input for each frame consists of a region of interest with irrelevant background removed. When no person is detected, the system does not generate keypoint prediction output, avoiding invalid calculations on empty frames. Pose tracking prediction runs under stable illumination and clean input conditions. The normalized coordinates and visibility of each keypoint in the output multi-view keypoint data are based on valid observations. Subsequent visibility-based fusion judgments are not distorted by illumination drift and background interference, and the reliability of visibility as a fusion weight is simultaneously improved. The combination of these two aspects makes the overall quality of the multi-view keypoint data surpass the effect of predicting directly on the entire frame.

[0021] S2. Transform the normalized coordinates of each key point in the multi-view key point data to the same reference coordinate system to obtain the reference coordinates of each key point in each view. When the visibility of a key point in at least one view is not lower than the first visibility threshold, fuse its reference coordinates in each view according to visibility to obtain the fused coordinates of the key point. When the visibility of a key point in each view is lower than the first visibility threshold, use its historical fused coordinates as the coordinates of the current frame within a preset number of supplementary frames. If the preset number of supplementary frames is exceeded, mark it as invisible and exit the posture determination. When the proportion of the number of key points in the temporal supplementation state exceeds a preset proportion threshold, determine that the determination result of the current frame is invalid and obtain the fused key point data.

[0022] Among them, the reference coordinate system is a three-dimensional coordinate system established based on the desktop calibration area; the fused coordinate is a single coordinate after the reference coordinates of the same key point in each view are fused according to visibility; the temporal supplementation state is the state in which the visibility of the key point in each view is lower than the first visibility threshold, and its historical fused coordinates are used as the coordinates of the current frame; the historical fused coordinate is the fused coordinate of the key point in previous frames; invisible is the state in which the key point is marked to be removed from the pose determination; the proportion of the number of key points in the temporal supplementation state is the proportion of the number of key points in the temporal supplementation state in the current frame to the total number of key points.

[0023] Specifically, the first visibility threshold is set to 0.3. For keypoints with visibility below 0.3, the coordinate prediction relies heavily on the model's spatial priors about adjacent keypoints rather than the observation of the keypoint itself. Directly accepting these priors would introduce systematic errors. The fusion is based solely on the comparison between the maximum difference in visibility of the keypoint across different viewpoints and a preset difference threshold. The comparison results are mutually exclusive and completely divided into two cases: when the maximum difference is less than the preset difference threshold of 0.2, the reference coordinates of the keypoint under each viewpoint are weighted and summed to obtain the fused coordinates, and the visibility of each viewpoint is weighted and summed and normalized to obtain the fused visibility of the keypoint; when the maximum difference is not less than 0.2, the reference coordinates of the keypoint under the viewpoint with the highest visibility are selected as the fused coordinates, and the visibility under that viewpoint is used as the fused visibility. This dual-branch fusion design addresses two failure modes of multi-view observation: When the observation quality between views is similar but generally average, if the views are switched independently with a fixed priority, the small differences in observation between views are amplified into coordinate jumps after the switch. The weighted summation branch ensures that the weights change continuously when the observation quality fluctuates between views, resulting in a smooth transition of the fused coordinates without any switching moments. When the observation quality between views is significantly different, the occlusion and blurring observation errors of the low-visibility view are mixed into the fused coordinates according to their weights, causing contamination. The optimal branch truncates the weight of the low-visibility view to zero, and its observation errors do not enter the fused coordinates. When the visibility is below the first visibility threshold, the historical fusion coordinates are used as the coordinates of the current frame within a preset number of supplementary frames K. K is taken as 30 frames at 30 frames per second, corresponding to the longest occlusion time of the arm or book on one side of the torso during reading, which is about 1 second. For the historical fusion coordinates that are still in use, a decreasing weight of 1-k / K is applied according to the number of frames k in which the key point has been continuously in the temporal supplementation state. In the early stage of supplementation, the reliability of the historical fusion coordinates is close to the current observation. It continues to decrease linearly with occlusion. When k equals K, the weight drops to zero. The contribution of the key point to the posture feature data is zero in the same frame. Then it is marked as invisible and the action of exiting the posture determination no longer changes the feature value. The exit process does not produce feature mutations. The posture classification model will not output category jitter due to the exit of the key point. The invisible mark at the key point level removes the single key point that has failed. After the bad point is removed at the point level, the remaining key points are maintained for determination. When the proportion of keypoints in the time-series supplementation state exceeds a preset threshold of 40%, the judgment result of the current frame is deemed invalid. If a large number of keypoints fail simultaneously, such as when a user leaves their seat or is heavily obscured by a book, frame-level protection refuses to output any judgment result. Abnormal posture judgments are only generated from frames with reliable observation. Point-level exit and frame-level circuit breaking constitute two levels of protection: single-point occlusion does not interrupt judgment, and multi-point failure does not output erroneous judgments. The reference coordinates of the same keypoint from different perspectives are comparable, and the fused coordinates change continuously with visibility. Posture judgment is not interrupted by single-viewpoint occlusion, and viewpoint switching does not cause coordinate jumps.

[0024] Furthermore, when the normalized coordinates are transformed to the same reference coordinate system, they are first multiplied by the width and height of the image at the viewpoint to restore the pixel coordinates, and then distortion correction is performed on the pixel coordinates. The distortion-corrected pixel coordinates are then inversely projected to the camera coordinate system. After determining the depth of the key points in the camera coordinate system, the coordinates in the camera coordinate system are transformed to the reference coordinate system through a rotation matrix and a translation vector to obtain the reference coordinates of each key point at each viewpoint.

[0025] The distortion correction is performed based on the intrinsic parameter matrix obtained from the calibration. The intrinsic parameter matrix includes the focal length, principal point, and radial distortion coefficients. The rotation matrix and translation vector constitute the extrinsic parameters of each viewpoint relative to the reference coordinate system. The calibration process is as follows: a checkerboard calibration object with 9×6 internal corner points and a grid side length of 25 mm is placed flat on the marked calibration area on the table. Each camera acquires no less than 15 images of the calibration object in different poses. Subpixel corner detection is performed on each image. The corner point image coordinates and the world coordinates established according to the calibration area form corresponding point pairs. The Zhang Zhengyou calibration method is used to solve the intrinsic parameter matrix and extrinsic parameters of each camera. The calibration results with a reprojection error root mean square exceeding 0.5 pixels are judged as invalid and require resampling. The recalibration process is also performed according to this procedure.

[0026] Specifically, after normalizing the coordinates and multiplying them by the width and height of the image at the given viewpoint to restore the pixel coordinates, distortion correction is applied to eliminate radial distortion of the lens. The results are then inversely projected onto the camera coordinate system using a pinhole imaging model. After determining the depth, extrinsic parameter transformation is performed to unify the coordinates to the reference coordinate system. This ordered process ensures that coordinates from different viewpoints are entered into the reference coordinate system with a unified numerical caliber. Combined with S2, the reference coordinates of each keypoint at each viewpoint are obtained through distortion correction and extrinsic parameter transformation using the same calibration benchmark. The reference coordinates of the same keypoint from any two viewpoints no longer contain systematic deviations caused by distortion and installation differences. Visibility fusion compares and weights coordinates with the same numerical caliber within the same coordinate system. The fusion result no longer superimposes the systematic errors of each viewpoint. The accuracy of the fused coordinates exceeds the effect of direct comparison and fusion in their respective original coordinate systems, providing a correct geometric premise for visibility as a fusion weight.

[0027] Furthermore, determining the depth of the key point in the camera coordinate system includes: when the baseline length of the two viewpoints is not less than a preset ratio of the shooting distance, triangulation is performed to recover the depth using the pixel coordinates of the same key point under different viewpoints as corresponding points; when the baseline length is less than the preset ratio or the uncertainty of the triangulated depth exceeds a preset ratio of the depth value, the intersection of the inverse projection ray and the preset constraint plane is taken as the depth, and the preset constraint plane is determined by edge fitting of the preset calibration area during the calibration stage.

[0028] Among them, the baseline is the line connecting the optical centers of the cameras from two different perspectives; triangulation is the process of recovering depth based on parallax, using the pixel coordinates of the same key point under different perspectives as corresponding points; the preset constraint plane is a vertical plane determined by fitting the edge of the desktop calibration area away from the reader, with the leading edge of the desktop as the reference; the preset proportion of depth uncertainty exceeding the depth value is 20% in this application, and the preset proportion of baseline length not less than the shooting distance is one-fifth in this application.

[0029] Specifically, the parallax condition is met when the baseline length is not less than one-fifth of the shooting distance, and the depth accuracy of the triangulation recovery is reliable. When the baseline is shorter or the depth uncertainty of the triangulation exceeds 20% of the depth value, the intersection of the back-projection ray and the preset constraint plane is taken as the depth. In a seated position, the key points of the upper body are approximately located within the neighborhood of this vertical plane. This plane constraint is only effective under the reading posture. If the user leans forward or backward significantly, causing the torso to deviate from this plane by more than ten centimeters, the deviation manifests as a change in proportional characteristics rather than a depth error, and does not affect the category classification of the posture. This selection mechanism allows the depth to be determined even with a small baseline arrangement, without needing to increase the camera spacing to obtain parallax. Combined with S2 and S3, the reliable determination of depth ensures that the third component of the reference coordinates of each key point under each viewpoint does not introduce large errors. The three-dimensional values ​​of the fused coordinates are reliable. The distance calculation on which the proportional features in the posture feature data depend is based on reliable three-dimensional coordinates. The accuracy of the determination of posture categories such as hand holding that depend on distance proportions improves synchronously with the reliability of depth. The combined effect of the two exceeds the ability to distinguish hand holding postures when constructing features only in a two-dimensional plane.

[0030] Furthermore, when the determination result of the current frame is deemed invalid, the system enters a circuit breaker state, suspends timing supplementation, and freezes the count update of the number of frames that have been continuously in the timing supplementation state for each key point. During the circuit breaker period, the proportion of key points in the timing supplementation state is continuously monitored. When the proportion remains below the preset proportion threshold within a preset number of consecutive recovery frames, the system exits the circuit breaker state, resumes timing supplementation, and resets the number of frames that have been continuously in the timing supplementation state to zero before restarting the count from the recovery frame.

[0031] Among them, the circuit breaker state is the state where the judgment result is invalid and all timing supplementation and count update are suspended; the count update is the accumulation of the number of frames that have been continuously in the timing supplementation state at each key point; the continuous preset recovery frame number is 20 frames in this application; clearing to zero and starting from the recovery frame again means that the frames that exited the circuit breaker state start to accumulate again.

[0032] Specifically, freezing the count during circuit breaker updates and resetting it to zero after recovery is a necessary construct to prevent recovery oscillations: if the number of frames that have been continuously in the timing supplementation state for each key point continues to be updated during the circuit breaker, the occlusion duration during the circuit breaker is still consuming the supplementation budget of each key point. At the moment of exiting the circuit breaker, most key points are close to or exceed K frames, and the circuit breaker is immediately triggered again. The system oscillates between circuit breaker and recovery and cannot output stably. Freezing the count ensures that the supplementation budget is not consumed during the circuit breaker, and the condition for exiting the circuit breaker is that the proportion within the continuously preset recovery frame number remains below the threshold. When this condition is met, it is confirmed that the occlusion has been lifted. Only when the budget is reset to zero and recalculated from the recovery frame does it correspond to the actual occlusion duration. Combined with S2, the frame-level determination of S2 is linked with the circuit breaker recovery mechanism. During the circuit breaker period, no determination results are output. After recovery, each key point retains a complete supplementary budget. Reliable determinations can be output in the early stage of recovery. The determination continuity after the end of long-term occlusion and the determination continuity of timing supplementation under short-term occlusion are uniformly guaranteed by the same budget mechanism. The combination of the two ensures that the system has no erroneous determination outputs in the full occlusion spectrum from short-term occlusion to long-term departure and no oscillations after recovery.

[0033] Furthermore, after obtaining the reference coordinates of each key point under each viewpoint, and before constructing the pose feature data from the frame coordinates of key points not marked as invisible, online drift verification is performed on the reference coordinates of each key point under each viewpoint. This verification is performed in parallel with subsequent steps. The pose time period is divided with the category switching event of the pose category data as the boundary. For key points corresponding to static body parts, in frames where the visibility of the key point under each viewpoint is higher than the second visibility threshold, the position deviation vector between the reference coordinates of the key point under different viewpoints is calculated. The position deviation vectors are statistically analyzed for the two pose time periods before and after, and the difference is used to obtain the residual deviation that eliminates the real body displacement. When the statistical quantity of the residual deviation on the sliding window exceeds the preset deviation threshold, it is determined that the transformation to the same reference coordinate system has drifted, the pose determination is paused and a recalibration prompt is output. After recalibration, the pose determination is resumed.

[0034] The posture time period is the time period between two adjacent category switches. If there are fewer than 10 valid samples in the time period, the time period is discarded. The key points corresponding to static body parts are based on the midpoint of the shoulder. The midpoint of the hip is only included if the side camera is installed at a height sufficient to observe the point over the edge of the table and its visibility meets the requirements. Whether or not it is included is automatically determined by the visibility of the point and is independent of the user's body size and table height. The second visibility threshold is set to 0.6 in this application. The sliding window length is 50 frames. The statistics are output only if there are no less than 30 valid samples in the window. The preset deviation threshold is set to five percent of the distance between the shoulders in the reference coordinate system.

[0035] Specifically, the reference coordinates of the same key point under different viewpoints are all in the reference coordinate system. If the transformation is error-free and there is no real displacement of the part, the two coincide, and the difference vector is the position deviation vector. The position deviation between viewpoints caused by attitude change changes with attitude switching. The deviation caused by calibration drift is a constant offset that does not disappear with attitude switching. The two types of deviations are mixed and indistinguishable in a single attitude period. After the deviation vector is differentially divided across attitude periods, the part that changes with attitude switching is eliminated, and the constant offset is retained as the residual deviation. The residual deviation therefore becomes the direct observation of calibration quality. Frames with visibility higher than 0.6 are included in the calculation to avoid spurious deviations introduced by low-quality observations themselves. The residual amount after normal attitude change is eliminated by differential is measured to be no more than 2% of the distance between the shoulders. The position offset corresponding to the minimum perceptible extrinsic parameter drift caused by the collision of the support structure is about 8% of the distance between the shoulders. The median of 5% is taken as the discrimination boundary. After the drift is determined, the attitude determination is paused and a recalibration prompt is output. After the user replaces the checkerboard calibration object according to the aforementioned camera calibration procedure to complete the recalibration, the attitude determination is resumed. Combined with S2, online drift verification ensures the continued reliability of the reference coordinate system during long-term operation. When drift occurs, the fused coordinates introduce a constant error due to the overall offset of the reference coordinates from each viewpoint. This verification intercepts the constant error before it enters the attitude determination and accumulates into a systematic misjudgment. After recovery, the fusion is re-established on an unbiased reference coordinate system. The combination of the two ensures that the long-term accuracy of visibility fusion does not degrade with slow disturbances such as support structure collisions and temperature deformation. The system can maintain the determination accuracy in the early stages of deployment without the need for periodic manual shutdown for calibration.

[0036] S3. Construct pose feature data from the frame coordinates of key points that are not marked as invisible in the fused key point data, and input the pose classification model to obtain pose category data.

[0037] The posture feature data consists of a feature set constructed from the frame coordinates of key points and the ratios between the coordinates. All features are normalized by the shoulder distance and are independent of the user's body shape. The posture classification model outputs six posture categories: normal, left-leaning, right-leaning, forward-leaning, backward-leaning, and hand-supporting. The classification model uses a random forest with 200 trees and a maximum depth of 12. When the categories are imbalanced, compensation is made according to the class weights. The macro average accuracy of cross-validation must be no less than 95% before deployment. In other implementations, support vector machines can be used instead of random forests. Training samples are collected under different lighting conditions, different rooms, and different users and labeled according to the six categories. There are no less than 600 frames per category for single-person samples. Hand-supporting samples are labeled with frames that have been verified by manual review based on detectable features. Boundary frames where the wrist is within ±10% of the judgment distance range are removed from the training set to avoid introducing labeling noise. The training set includes negative samples with multiple people in the frame and under insufficient lighting conditions to suppress misjudgment of the companion's posture as the target posture.

[0038] Specifically, when frame coordinates are used as fusion coordinates in valid observation frames and as historical fusion coordinates with decreasing weights in temporal supplementary frames, the meaning of their reference coordinate systems remains consistent. Features are constructed using key points not marked as invisible, and input into the pose classification model to obtain pose category data. Shoulder distance normalization ensures that feature values ​​do not change with the user's shoulder width or sitting height, and similar poses of users with different body types overlap in the feature space. The generalization ability of training samples is not limited by the body type distribution of the collected objects. Removal of boundary frames for hand-holding categories avoids annotation noise from entering the model, and the inclusion of negative samples suppresses misjudgments when multiple people are in the frame. Frame-level input allows pose category data to be generated at a frame-level frequency, providing temporal resolution for subsequent reminder trigger determination.

[0039] Furthermore, the constructed posture feature data includes: features constructed from the current frame coordinates and coordinate ratios of key points in the head, shoulders, elbows, wrists and upper spine that are not marked as invisible. Each feature is normalized by the shoulder distance to make the posture feature data independent of body shape. Among them, the current frame coordinates are fused coordinates in the effective observation frames and historical fused coordinates with decreasing weights in the time-series supplementary frames.

[0040] The posture feature data comprises 66 features: 22 coordinate features, representing the horizontal and vertical components of the current frame coordinates of 11 key points; and 22 angle features, including the angle between the line connecting the left and right shoulders and the horizontal direction (its sign corresponds to the left and right tilt direction), the angle between the line connecting the neck and the midpoint of the shoulders and the vertical direction (used to distinguish between forward and backward tilt), the angle between the lines connecting the left and right upper arms and the torso, the angle between the left and right forearms and the horizontal direction, and their inter-frame variations. Each angle is obtained by performing a dot product and arctangent operation on the vector of the key point's current frame coordinates. The inter-frame variation is taken as the absolute value of the difference between the current frame feature value and the corresponding feature value in the previous frame, multiplied by the distance between the shoulders. Normalization; 22 proportional features, including the ratio of the distance between the head key point and the midpoint of the shoulder to the distance between the shoulders, used to quantify the degree of forward tilt, the ratio of the distance from the wrist key point to the elbow to the shoulder, the ratio of the torso height to the distance between the shoulders, and their combinations; the hand-supporting posture refers to the state in which the user supports their chin or cheek with one or both hands, or supports the weight of their head on the table with their elbows. Its detectable features are that the wrist key point is located within a preset distance below the chin or cheek key point, and the height of the elbow key point is lower than the shoulder key point and close to the height of the table. Among the proportional features, the ratio of the distance from the wrist key point to the elbow to the shoulder is used to quantify this posture.

[0041] Specifically, the distinction between the six pose types directly relies on a combination of angle and proportion features; a single type of coordinate feature alone cannot achieve this distinction: left and right tilts are distinguished by the sign of the shoulder line angle, forward and backward tilts are distinguished by the ratio of the angle between the neck line and the head distance, and hand-supporting is distinguished by the ratio of the wrist and elbow distance. Combined with S3 and S2, the coordinates of the current frame in the temporal supplement frame are historical fusion coordinates with decreasing weights. Feature values ​​transition smoothly during occlusion rather than jumping or becoming invalid. The inter-frame change features remain continuous during occlusion, and the pose classification model does not experience pseudo-class switching due to temporal supplementation. Keypoints marked as invisible only exit feature construction after their weights decay to zero. The feature dimension remains consistent before and after occlusion, the input dimension of the classification model is stable, the model does not require separate training for occlusion scenarios, and the occluded state and the normal observation state share the same classification decision boundary. The combination of these two aspects ensures that the accuracy of the classification model in occluded scenarios remains consistent with that in normal observation scenarios.

[0042] S4. Perform reminder trigger determination and weighted statistics on the posture category data to obtain training guidance data.

[0043] The process includes: reminder trigger judgment, which is the process of triggering and stopping the judgment when the posture category data belongs to the preset abnormal posture category; weighted statistics, which is the statistical process after the judgment results are graded and weighted; training guidance data, which is the training guidance given for the corresponding category of the long-term abnormal posture in the suggestion report data; historical statistics, which are the statistics of each posture category in the previous statistical period; and statistically analyzing the statistics of each posture category within a preset time. When the statistics of the abnormal posture category exceed the preset statistical threshold, suggestion report data containing habit identification data and training guidance data is generated.

[0044] Specifically, the reminder trigger determination adopts trigger conditions and stop conditions in conjunction with minimum interval adjustment: the trigger condition is that when the posture category data belongs to a preset abnormal posture category and the number of consecutive frames in a single instance reaches the first continuous frame threshold, reminder data corresponding to the preset abnormal posture category is output. In this application, the reminder data is a voice reminder; the stop condition is that when the posture category data no longer belongs to the preset abnormal posture category during the reminder data output period and the number of consecutive frames in a single instance reaches the second continuous frame threshold, the reminder data output is stopped, creating a hysteresis between triggering and stopping; the minimum interval between two adjacent reminder data outputs is adaptively adjusted according to historical statistics. Instant reminders are responsible for the immediate intervention of newly emerging occasional anomalies, while long-term correction is undertaken by the training guidance data in the suggestion report data. The two are automatically divided according to the statistical results; the time period when the determination result is invalid is not included in the statistics, so that the habit identification data and training guidance data are not contaminated by occlusion time periods. The obtained historical statistics are used to adjust the interval of the reminder data. The combination of stricter-than-stop triggers and adaptive intervals ensures that users receive timely reminders in the early stages of correction. Once a habit is formed, the reminder frequency increases as the percentage decreases and decreases as the percentage increases. The reminder strategy automatically evolves with the user's correction process. Statistics and reminders operate under the same category system, requiring no manual configuration.

[0045] Furthermore, the reminder trigger determination includes a trigger condition and a stop condition. The trigger condition is that when the posture category data belongs to a preset abnormal posture category and the number of consecutive frames in a single instance reaches a first continuous frame threshold, reminder data corresponding to the preset abnormal posture category is output. The stop condition is that during the reminder data output period, when the number of consecutive frames in a single instance where the posture category data no longer belongs to the preset abnormal posture category reaches a second continuous frame threshold greater than the first continuous frame threshold, the reminder data output is stopped, creating a hysteresis between triggering and stopping. The number of consecutive frames in a single instance targeted by the trigger condition and the stop condition is reset to zero and recounted when the posture category deviates from the currently targeted preset abnormal posture category.

[0046] In this application, the first continuous frame threshold is set to 5 frames; the second continuous frame threshold is set to 15 frames. The trigger counter counts the number of consecutive continuous frames for the target abnormal category. If the frame deviates from the target category midway, the counter is reset and recounted. The alert is output only when the cumulative total reaches 5 frames. The stop counter counts the number of consecutive continuous frames for the return to the normal category. If the frame falls back into the abnormal category midway, the counter is reset and recounted. The alert is stopped only when the cumulative total reaches 15 frames. The counting object is the number of consecutive continuous frames for a single time period, not the cumulative number of frames across time periods.

[0047] Specifically, jitter near the category boundary manifests as short, continuous segments alternating between the two categories. In practice, the duration of a single jitter segment typically does not exceed three frames. Both thresholds exceed the longest jitter duration and have different values. The trigger condition is stricter than the stop condition. Once triggered, the alert remains active during the boundary jitter period without repeated restarts due to jitter. Resetting and recounting ensures no residual counts during jitter, preventing the accumulation of short, continuous segments into false durations spanning different time periods. Combined with S4, the alert data output by S4 is generated only after confirmation of a single consecutive continuous frame count. Hysteresis ensures different trigger and stop criteria, preventing boundary jitter from being converted into repeated restarts of the alert signal. Each alert heard by the user corresponds to a confirmed abnormal behavior rather than boundary noise. The credibility of the alert and the user's willingness to comply are simultaneously enhanced. The combination of these two factors makes the corrective effect of the alert greater than the effect achieved by simply lowering the threshold for frequent alerts.

[0048] Furthermore, the minimum interval between two adjacent output reminder data is adaptively adjusted based on historical statistics. The historical statistics are the proportion of the same preset abnormal posture category in the previous statistical period. The minimum interval is determined by a monotonic mapping function. The larger the proportion, the longer the minimum interval. There is no cold start phase without this proportion. The minimum interval is taken as the lower limit of the monotonic mapping function. The parameters of the monotonic mapping function are constrained by the reminder correction effectiveness retention rate, so that the number of effective reminders per unit time after adjustment is not less than the preset proportion before adjustment.

[0049] Among them, the historical statistic is the proportion p of the duration of the abnormal posture category in the previous statistical period, with a value range of 0 to 1; the monotonic mapping function in this application is a piecewise linear function: when p is not greater than 0.3, the minimum interval T is 30 seconds; when p is greater than 0.3 and not greater than 0.7, T is 30 seconds plus (p-0.3) / (0.7-0.3)×270 seconds; when p is greater than 0.7, T is 300 seconds; the cold start stage is the stage where there is no complete statistical period and no historical proportion, and the minimum reminder interval for each abnormal category is the lower limit of the mapping function, 30 seconds; the reminder correction effectiveness retention rate is the number of effective reminders per unit time after frequency reduction, that is, the proportion of the user correcting the posture within the second continuous frame threshold after triggering, and the preset proportion is 70% in this application.

[0050] Specifically, the larger the proportion, the longer the minimum interval. Habitually stable abnormal postures are not frequently reminded due to their continued existence. During the cold start phase, timely intervention is obtained with the minimum interval. After the first statistical period ends and historical proportions are generated, adaptive adjustment begins. The parameters of the mapping function are constrained by the reminder correction effectiveness retention rate. After the frequency is reduced, the number of effective reminders per unit time is not less than 70% of the number before the frequency reduction. This constraint limits the frequency reduction range to a range that does not lose the correction effectiveness, and the frequency reduction will not slide into laissez-faire. Combined with the hysteresis trigger of S4, the hysteresis ensures that each triggered alert corresponds to a confirmed anomaly. The minimum interval is adjusted based on this reliable trigger. The statistical object and the trigger object are both abnormal postures confirmed after a single continuous frame. Therefore, the proportion p truly reflects the occurrence level of this type of anomaly, and the frequency reduction is based on reliability. If the trigger itself contains jitter noise, the proportion is contaminated by noise, and the frequency reduction will mistakenly suppress the alert of the real anomaly. The combination of the two makes the balance between alert fatigue and correction effectiveness achieved by frequency reduction based on reliable trigger and reliable statistics. The retention rate of the effective correction intensity after frequency reduction compared with that before frequency reduction is explicitly constrained. The overall correction effect is better than either fixed interval alert or unconstrained frequency reduction alone.

[0051] Furthermore, the weighted statistics include: frames with invalid judgment results are not included in the statistics; frames with a proportion of key points in the time-series supplementation state that exceeds a preset weight reduction threshold but does not exceed the preset proportion threshold are included in the statistics with a weight lower than that of normal frames; and the remaining frames are included in the statistics with full weight. The weighted statistics are used for habit determination.

[0052] Among them, the preset weighting threshold and weighting weight can be preset according to the impact of the degree of occlusion on the statistical results; normal frames are frames included in the statistics with full weight; the habit judgment is based on the weighted statistics.

[0053] Specifically, the three-tier weighting explicitly models the relationship between occlusion degree and statistical reliability: frames with invalid judgment results are not included in the statistics at all; frames with a proportion of key points in the temporal supplementation state exceeding the preset weighting threshold but not exceeding the preset proportion threshold are weighted down because their coordinates partially rely on historical data; the remaining frames are included with full weight. The weighted statistics are used for habit determination, and habit identification data and training guidance data are not contaminated by occlusion periods. Combined with S2, the circuit breaker judgment of S2 and the three-tier statistics share the same proportion index: frames with a proportion exceeding the preset proportion threshold are triggered by circuit breaking and given zero weight; frames with a proportion between the preset weighting threshold and the preset proportion threshold are weighted down and included; frames with a proportion below the preset weighting threshold are given full weight. The occlusion degree is continuously graded by the same index at both the judgment and statistics ends. The judgment end rejects outputting unreliable results, and the statistics end calculates the weight according to the degree of unreliability. The combination of the two systematically compresses the deviation between the statistics corresponding to habit identification data and training guidance data in the suggestion report data and the actual pose distribution. The report has higher guidance value for long-term correction than the report under unweighted statistics.

[0054] In one specific implementation: posture classification is performed on the host computer at a frame-level frequency. The host computer only outputs the posture category as a single-byte category code to the slave computer via the serial port. The high four bits of the category code are the posture category number, and the low four bits are the reminder trigger flag. The slave computer adopts a microcontroller board with 54 digital input / output ports, 16 analog input ports, 4 serial ports and a 16 MHz crystal oscillator. The slave computer is connected to a voice playback device. After receiving the category code with the reminder trigger flag set, it plays the corresponding category's voice reminder.

[0055] In one specific implementation: the present invention also proposes a posture recognition system based on multi-view coordinate fusion, comprising: an image acquisition module, used to perform posture tracking and prediction on multi-view image data to obtain multi-view key point data; a coordinate unification and fusion module, used to transform the normalized coordinates of each key point in the multi-view key point data under each viewpoint to the same reference coordinate system, fuse them according to visibility, supplement them temporally, and perform circuit breaker protection to obtain fused key point data; a posture classification module, used to construct posture feature data from the current frame coordinates of key points not marked as invisible in the fused key point data, input them into a posture classification model to obtain posture category data; and a reminder and statistics module, used to perform reminder trigger determination and weighted statistics on the posture category data to obtain training guidance data, and feed back historical statistics to the reminder trigger determination to adjust the reminder interval; the modules transmit the above data objects through a bus and a call interface, and the host computer and the slave computer transmit the category code through a serial port.

[0056] In one specific embodiment: the present invention also proposes a computer-readable storage medium on which a computer program is stored. The computer program is stored on the computer-readable storage medium in the form of a sequence of instructions that can be read and executed by a processor. When the processor reads and executes the sequence of instructions, it implements each step of the above-mentioned attitude recognition method based on multi-view coordinate fusion one by one. The computer-readable storage medium can be a non-volatile or volatile storage medium such as a read-only memory, random access memory, magnetic disk, optical disk, or portable hard drive.

[0057] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A pose recognition method based on multi-view coordinate fusion, characterized in that, include: S1. Perform pose tracking and prediction on multi-view image data to obtain multi-view keypoint data; S2. Transform the normalized coordinates of each key point in the multi-view key point data to the same reference coordinate system to obtain the reference coordinates of each key point in each view. When the visibility of a key point in at least one view is not lower than the first visibility threshold, fuse its reference coordinates in each view according to visibility to obtain the fused coordinates of the key point. When the visibility of a key point in each view is lower than the first visibility threshold, use its historical fused coordinates as the coordinates of the current frame within a preset number of supplementary frames. If the preset number of supplementary frames is exceeded, mark it as invisible and exit the posture determination. When the proportion of the number of key points in the temporal supplementation state exceeds a preset proportion threshold, determine that the determination result of the current frame is invalid and obtain the fused key point data. S3. Construct pose feature data from the frame coordinates of key points that are not marked as invisible in the fused key point data, and input the pose classification model to obtain pose category data. S4. Perform reminder trigger determination and weighted statistics on the posture category data to obtain training guidance data.

2. The pose recognition method based on multi-view coordinate fusion according to claim 1, characterized in that, The pose tracking prediction includes: performing illumination normalization preprocessing and person detection on each frame of multi-view image data; waiting when no person is detected; locating the region of interest containing the person when a person is detected; and predicting the normalized coordinates and visibility of each key point frame by frame from the image data within the region of interest to obtain multi-view key point data.

3. The pose recognition method based on multi-view coordinate fusion according to claim 2, characterized in that, When the normalized coordinates are transformed to the same reference coordinate system, they are first multiplied by the width and height of the image at the viewpoint to restore the pixel coordinates. Then, distortion correction is performed on the pixel coordinates. The distortion-corrected pixel coordinates are inversely projected to the camera coordinate system. After determining the depth of the key points in the camera coordinate system, the coordinates in the camera coordinate system are transformed to the reference coordinate system through a rotation matrix and a translation vector to obtain the reference coordinates of each key point at each viewpoint.

4. The pose recognition method based on multi-view coordinate fusion according to claim 3, characterized in that, The determination of the depth of the key point in the camera coordinate system includes: when the baseline length of the two viewpoints is not less than a preset ratio of the shooting distance, triangulation is performed to recover the depth using the pixel coordinates of the same key point under different viewpoints as corresponding points; when the baseline length is less than the preset ratio or the uncertainty of the triangulation depth exceeds a preset ratio of the depth value, the intersection of the inverse projection ray and the preset constraint plane is taken as the depth, and the preset constraint plane is determined by the edge fitting of the preset calibration area during the calibration stage.

5. The pose recognition method based on multi-view coordinate fusion according to claim 1, characterized in that, When the determination result of the current frame is invalid, the system enters the circuit breaker state, pauses timing supplementation, and freezes the count update of the number of frames that have been continuously in the timing supplementation state for each key point. During the circuit breaker period, the system continuously monitors the proportion of key points in the timing supplementation state. When the proportion remains below the preset proportion threshold within a preset number of consecutive recovery frames, the system exits the circuit breaker state, resumes timing supplementation, and resets the number of frames that have been continuously in the timing supplementation state to zero before restarting the count from the recovery frame.

6. The pose recognition method based on multi-view coordinate fusion according to claim 1, characterized in that, The constructed posture feature data includes: features constructed from the current frame coordinates and the ratios between coordinates of key points in the head, shoulders, elbows, wrists and upper spine that are not marked as invisible. Each feature is normalized by the distance between the shoulders, so that the posture feature data is independent of body shape. Among them, the current frame coordinates are fused coordinates in the effective observation frames and historical fused coordinates with decreasing weights in the time-series supplementary frames.

7. The pose recognition method based on multi-view coordinate fusion according to claim 1, characterized in that, The reminder trigger determination includes a trigger condition and a stop condition; the trigger condition is that when the posture category data belongs to a preset abnormal posture category and the number of consecutive frames in a single instance reaches the first continuous frame threshold, reminder data corresponding to the preset abnormal posture category is output. The stopping condition is that during the output of the reminder data, when the number of consecutive frames in which the posture category data no longer belongs to the preset abnormal posture category reaches a second consecutive frame threshold that is greater than the first consecutive frame threshold, the output of the reminder data is stopped, so that there is a hysteresis between the trigger and the stop. The number of consecutive frames in which the trigger condition and the stop condition are applied is reset to zero and recounted when the posture category deviates from the currently applied preset abnormal posture category.

8. The pose recognition method based on multi-view coordinate fusion according to claim 7, characterized in that, The minimum interval between two consecutive output reminder data is adaptively adjusted based on historical statistics. The historical statistics are the proportion of the same preset abnormal posture category in the previous statistical period. The minimum interval is determined by a monotonic mapping function. The larger the proportion, the longer the minimum interval. There is no cold start phase without this proportion. The minimum interval is taken as the lower limit of the monotonic mapping function. The parameters of the monotonic mapping function are constrained by the reminder correction effectiveness retention rate, so that the number of effective reminders per unit time after adjustment is not less than the preset proportion before adjustment.

9. The pose recognition method based on multi-view coordinate fusion according to claim 1, characterized in that, The weighted statistics include: frames with invalid results are not included in the statistics; frames with a proportion of key points in the time-series supplementation state that exceeds the preset weight reduction threshold but does not exceed the preset proportion threshold are included in the statistics with a weight lower than that of normal frames; and the remaining frames are included in the statistics with full weight. The weighted statistics are used for habit determination.

10. The pose recognition method based on multi-view coordinate fusion according to claim 1, characterized in that, After obtaining the reference coordinates of each key point in each viewpoint, and before constructing the pose feature data from the frame coordinates of key points not marked as invisible, online drift verification is performed on the reference coordinates of each key point in each viewpoint. This verification is performed in parallel with subsequent steps. The pose time period is divided with the category switching event of the pose category data as the boundary. For key points corresponding to static body parts, in frames where the visibility of the key point in each viewpoint is higher than the second visibility threshold, the position deviation vector between the reference coordinates of the key point in different viewpoints is calculated. The position deviation vectors are statistically analyzed for the two pose time periods before and after, and the difference is used to obtain the residual deviation that eliminates the real body displacement. When the statistical quantity of the residual deviation on the sliding window exceeds the preset deviation threshold, it is determined that the transformation to the same reference coordinate system has drifted. The pose determination is paused and a recalibration prompt is output. The pose determination is resumed after recalibration.

Images