Training action trajectory deviation monitoring method and device based on human key point tracking

Abstract

The application discloses a training action track deviation monitoring method and device based on human key point tracking, and belongs to the field of data processing.The method comprises the following steps: acquiring a training image sequence, performing human key point tracking to obtain a key point track; matching a standard action track to obtain an action track deviation; performing time sequence compensation on the human key point track, determining a target acquisition delay amount according to a track recovery feature; in response to the action track deviation meeting a feedback condition, outputting action correction information through a display, acquiring a corresponding correction response track, and determining a target display delay amount according to a historical correction response baseline and the correction response track; and monitoring the deviation source of the action track deviation to obtain an image acquisition delay deviation, a display feedback delay deviation and a human action deviation.The application solves the technical problems that the prior art cannot disassemble mixed action track deviations, and it is difficult to monitor and distinguish the deviation sources.

Description

Technical Field

[0001] This invention relates to the field of data processing, and specifically to a method and device for monitoring training motion trajectory deviations based on human body key point tracking. Background Technology

[0002] In fitness training, rehabilitation training, and other scenarios, motion trajectory deviation monitoring based on visual tracking of human key points has become a mainstream technical means to assist in motion correction and evaluate training quality. However, when calculating motion deviation, existing related technologies usually directly compare the human key point trajectory obtained from image acquisition with the standard motion trajectory. This does not fully consider the timing delay at the acquisition end introduced throughout the image acquisition, signal transmission, and key point recognition algorithm processing, as well as the timing delay at the display feedback end generated during the rendering and refresh of correction information on the display and the perception of the training object. It is impossible to decompose the sources of mixed motion trajectory deviations. This can easily lead to false deviations caused by inherent delays in the system hardware links being misjudged as motion execution deviations of the training object, resulting in distorted motion quality evaluation. It is also difficult to achieve fine monitoring and differentiation of the sources of deviations, and cannot meet the actual needs of high-precision motion correction and professional training quality evaluation. Summary of the Invention

[0003] This application provides a method and device for monitoring training motion trajectory deviations based on human keypoint tracking, aiming to solve the technical problem that existing technologies cannot decompose the sources of mixed motion trajectory deviations and are difficult to monitor and distinguish the sources of deviations.

[0004] In view of the above problems, this application provides a method and device for monitoring training motion trajectory deviation based on human key point tracking.

[0005] The first aspect disclosed in this application provides a method for monitoring training motion trajectory deviation based on human keypoint tracking, the method comprising: The process involves: acquiring a training image sequence corresponding to the target training action performed by the training subject; tracking human keypoints in the training image sequence to obtain human keypoint trajectories; matching the human keypoint trajectories with the standard action trajectories corresponding to the target training action to obtain action trajectory deviations; determining multiple candidate acquisition delays based on the image acquisition cycle of the image sensor; performing time-series compensation on the human keypoint trajectories according to each candidate acquisition delay; determining the target acquisition delay based on the trajectory recovery characteristics of the multiple target human keypoints after compensation; outputting action correction information through a display in response to the action trajectory deviation meeting feedback conditions; acquiring the corresponding correction response trajectory; determining the target display delay based on the historical correction response baseline and the correction response trajectory; and monitoring the source of the action trajectory deviation based on the target acquisition delay and the target display delay to obtain image acquisition delay deviation, display feedback delay deviation, and human action deviation.

[0006] Another aspect of this application discloses a training motion trajectory deviation monitoring device based on human keypoint tracking, the device comprising: The system includes a tracking module for acquiring training image sequences corresponding to the target training action performed by the training object, tracking human key points in the training image sequences to obtain human key point trajectories; a matching module for matching the human key point trajectories with standard action trajectories corresponding to the target training action to obtain action trajectory deviations; a compensation module for determining multiple candidate acquisition delays based on the image acquisition cycle of the image sensor, performing time-series compensation on the human key point trajectories according to each candidate acquisition delay, and determining the target acquisition delay based on the trajectory recovery characteristics of the multiple target human key points after compensation; a correction module for outputting action correction information through a display in response to the action trajectory deviation meeting feedback conditions, acquiring the corresponding correction response trajectory, and determining the target display delay based on the historical correction response baseline and the correction response trajectory; and a monitoring module for monitoring the source of the action trajectory deviation based on the target acquisition delay and the target display delay to obtain image acquisition delay deviation, display feedback delay deviation, and human action deviation.

[0007] One or more technical solutions provided in this application have at least the following technical effects or advantages: By acquiring training image sequences of the training subjects performing the target training actions and tracking the trajectory of human key points, the overall motion trajectory deviation is obtained by matching these deviations with the standard motion trajectory. Multiple candidate acquisition delays are then set with the image sensor acquisition cycle as the smallest granularity. The target acquisition delay is determined iteratively by combining the trajectory recovery features of multiple target key points after time-compensation. Simultaneously, in response to deviations meeting feedback conditions, motion correction information is output and the correction response trajectory is acquired. The target display delay is calculated by combining the historical correction response baseline that matches the current motion stage and correction direction. Finally, based on these two types of delays, the sources of overall motion trajectory deviation are separated and monitored, resulting in three independent components: image acquisition delay deviation, display feedback delay deviation, and human motion deviation. Without requiring additional hardware acquisition equipment, this effectively eliminates the interference of inherent system link delays on motion deviation determination, improving the accuracy of motion trajectory deviation assessment and the precision of deviation source localization.

[0008] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0009] Figure 1 A flowchart illustrating a method for monitoring training motion trajectory deviation based on human keypoint tracking is provided for embodiments of this application. Figure 2 A schematic diagram of the structure of a training motion trajectory deviation monitoring device based on human key point tracking is provided for the embodiments of this application.

[0010] Explanation of reference numerals in the attached diagram: Tracking module 11, Matching module 12, Compensation module 13, Correction module 14, Monitoring module 15. Detailed Implementation

[0011] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0012] The overall concept of the technical solution provided in this application is as follows: This application provides a method and device for monitoring training motion trajectory deviation based on human keypoint tracking. By acquiring training image sequences and performing human keypoint tracking, the trajectory of human keypoints is obtained. This trajectory is then matched with a standard motion trajectory to obtain the overall motion trajectory deviation. Multiple candidate acquisition delay values ​​are set at the image acquisition cycle granularity. The target acquisition delay value is determined iteratively by combining the trajectory recovery features after time-compensated analysis. Simultaneously, when correction feedback is triggered, the correction response trajectory is acquired. The target display delay value is calculated by matching the historical correction response baseline of the training object. Finally, the overall motion trajectory deviation is decomposed based on the two types of delay values, quantifying the image acquisition delay deviation, display feedback delay deviation, and human motion deviation. Without requiring additional hardware acquisition equipment, the inherent system-side delay and human execution deviation are separated, effectively eliminating time-series interference in the acquisition and display links.

[0013] After introducing the basic principles of this application, various non-limiting embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0014] Example 1, as Figure 1 As shown in the embodiment of this application, a method for monitoring training motion trajectory deviation based on human keypoint tracking is provided. The method includes: S100: Obtain the training image sequence corresponding to the target training action performed by the training object, and perform human key point tracking on the training image sequence to obtain the human key point trajectory.

[0015] Specifically, firstly, image sensors deployed in the training scene continuously sample the training scene according to a pre-set image acquisition cycle, capturing images frame by frame throughout the entire process of the training object performing the target training action. Finally, a training image sequence is obtained, ordered by the acquisition timestamp and covering the complete cycle of the action from the starting posture to the ending posture. Here, the training object refers to the main user who actually performs the training action, such as fitness trainees or rehabilitation patients, and is the entity that carries and generates the movement deviation. The target training action is a specific training action with a pre-set standard posture sequence, movement path, and temporal rhythm. The training image sequence is a collection of multiple consecutive images carrying precise acquisition timestamps. Each frame corresponds to an instantaneous posture during the execution of the action, completely preserving the most original spatial form and temporal node information of the action, which is the original data source for all subsequent calculations.

[0016] After acquiring the training image sequence, the system performs human keypoint tracking on the sequence. First, for each frame of the image, a deep learning-based human pose estimation algorithm is used to identify multiple preset key skeletal nodes on the human body, including core joint nodes such as acromion, elbow joint, wrist joint, hip joint, knee joint, and ankle joint, as well as trunk reference points. The spatial coordinates of each node in a single frame image are output. Then, based on the motion continuity constraints between frames and the geometric constraints of the human skeleton, the keypoints identified in adjacent frames are matched and their links are spliced ​​to ensure that the same physical keypoint is continuously tracked in different frames, avoiding trajectory breaks or offsets caused by keypoint identity mismatch between frames. This complete processing process from single-frame keypoint identification to cross-frame continuous association is human keypoint tracking. Human keypoint tracking is the core processing action that transforms the unstructured visual information of the original image into structured motion data. Its core value lies in filtering irrelevant visual interference such as background, clothing, and lighting, retaining only the node position information directly related to limb movement, while ensuring the consistency of the identity of the same keypoint throughout the entire sequence.

[0017] After completing the full sequence tracking, the spatial coordinates of each keypoint in all frames are concatenated in chronological order of acquisition time to form the spatial position change path of the keypoint over time. The position change paths of all keypoints together constitute the human keypoint trajectory. The human keypoint trajectory is a structured digital representation of the training object's action execution process. Each single keypoint trajectory contains both spatial coordinate dimensions and time dimensions. It can not only intuitively reflect the movement path, movement amplitude, and posture angle changes of the corresponding limb, but also correspond to different time stages of action execution through timestamps. It is the core data carrier for all subsequent deviation calculations, delay compensation, and deviation source decomposition.

[0018] S200: Match the trajectory of the human body key points with the standard motion trajectory corresponding to the target training motion to obtain the motion trajectory deviation.

[0019] Specifically, the first step is to retrieve the standard motion trajectory that corresponds one-to-one with the target training motion. The standard motion trajectory is a baseline trajectory data that is pre-generated by professionals who perform standard motions and uses the same key point definition and spatial coordinate system as the actual trajectory. It includes the standard spatial motion path of each key point in the complete process of the standard motion, and also marks the standard time sequence nodes corresponding to each stage of the motion. It is a unified reference benchmark for measuring the degree of deviation of the actual motion.

[0020] The system then performs a matching operation between the human body key point trajectory and the standard motion trajectory. This matching operation does not directly perform a frame-by-frame hard alignment of the two trajectories. Instead, it first extracts multiple motion temporal feature nodes from the two trajectories. These motion temporal feature nodes refer to iconic posture nodes with clear phase identification significance during the execution of the motion, such as the starting posture point, the peak point of the motion amplitude, the limb movement reversal point, and the ending posture point. These nodes correspond to the key phase boundaries of the motion and have the characteristics of high posture recognition and clear temporal positioning.

[0021] After feature node extraction is completed, the system establishes the node correspondence between the corresponding action timing feature nodes in the two trajectories based on the semantic consistency of the action phase. That is, the feature nodes representing the same action stage in the actual trajectory are paired one by one with the same-name feature nodes in the standard trajectory to ensure that subsequent deviation calculations always target the same execution phase of the action.

[0022] Based on this, the system calculates the temporal and spatial differences between each group of feature nodes with corresponding relationships. The temporal difference refers to the time difference between the occurrence time of the same feature node in the actual trajectory and the standard time in the standard trajectory, which is used to reflect the deviation in the speed of the action execution. The spatial difference refers to the spatial coordinate difference of the corresponding human key points under the same temporal phase, which is used to reflect the degree of spatial deviation of the actual position of the limb relative to the standard position. The system integrates and calculates the temporal and spatial differences of all feature nodes in the entire action process, and finally obtains the action trajectory deviation covering the entire cycle of the action. This deviation is a comprehensive quantitative result that includes both temporal rhythm deviation and spatial path deviation.

[0023] S300: Determine multiple candidate acquisition delays based on the image acquisition cycle of the image sensor, perform time-series compensation on the trajectory of the human body key points according to each of the candidate acquisition delays, and determine the target acquisition delay based on the trajectory recovery features of the multiple target human body key points after compensation.

[0024] Specifically, the acquisition delay search interval is first defined based on the temporal difference between the corresponding action temporal feature nodes obtained by matching the preceding action trajectory. The acquisition delay search interval is a time range extending around the estimated acquisition delay amplitude, used to limit the search boundary of the candidate delay. Its value is taken as the center of the overall temporal difference of the feature nodes and expands to both sides within a reasonable range, thus covering the possible range of the actual acquisition delay.

[0025] Based on this, multiple candidate acquisition delay values ​​are generated by combining the image acquisition cycle of the image sensor. The image acquisition cycle is the inherent time interval for the image sensor to complete the acquisition of one frame, representing the time granularity of sampling and the smallest adjustment unit for timing compensation. After uniformly and discretely dividing the acquisition delay search interval according to this acquisition cycle, each time value obtained is a candidate acquisition delay value. Each candidate acquisition delay value is an integer multiple of the acquisition cycle, representing a set of alternative values ​​for time lag caused by image acquisition, transmission, and key point recognition processing to be verified. Subsequently, the system performs timing compensation operations on the human key point trajectory according to each candidate acquisition delay value. Specifically, timing compensation involves shifting the time axis of the entire human key point trajectory forward by the duration of the corresponding candidate delay value, that is, subtracting the candidate acquisition delay value from the acquisition time corresponding to the key point coordinates of each frame. This simulates the process of offsetting the acquisition lag effect and restoring the trajectory shape corresponding to the actual occurrence time of the action. After each shift of a candidate delay value is completed, a set of timing-compensated target human key point trajectories is obtained.

[0026] For each compensated trajectory, the system extracts trajectory recovery features from multiple target human key points. These target human key points are pre-selected core skeletal nodes that exhibit stable motion patterns during the target training movements and are minimally affected by subjective movement deformations. Typically, nodes with strong skeletal geometric constraints, such as the torso reference point and bilateral hip and knee joints, are selected as reference benchmarks for judging the temporal rationality of the trajectory. The trajectory recovery features specifically include two core indicators: temporal recovery degree and relative geometric stability. The temporal recovery degree refers to the similarity in temporal distribution between the temporal feature nodes of the compensated trajectory and the corresponding feature nodes of the standard movement trajectory. The degree of temporal recovery is used to measure whether the occurrence time of each phase of the compensated action is aligned with the rhythm of the standard action. The higher the degree of temporal recovery, the closer the temporal rhythm of the trajectory after the delay compensation is to the actual action rhythm. The degree of relative geometric stability refers to the degree of fluctuation of the relative skeletal geometric position relationship between multiple target key points between consecutive frames after compensation. Based on the physical constraints of fixed bone length and smooth change of joint angle in real human motion, if the temporal misalignment is caused by abnormal jumps in the bone ratio between frames, the higher the degree of relative geometric stability, the more the spatial geometric relationship of the compensated trajectory conforms to the physical laws of human motion.

[0027] The system further weights and fuses the temporal recovery degree with the relative geometric stability degree to calculate the acquisition recovery degree of the corresponding candidate acquisition delay. The acquisition recovery degree is a comprehensive quantitative index that evaluates the temporal and geometric rationality of the trajectory after compensation for the candidate delay. The higher the value, the closer the candidate delay is to the real image acquisition delay. After calculating all candidate delays, the system arranges the corresponding acquisition recovery degrees in order of their numerical values ​​to form an acquisition recovery degree sequence. Then, it selects the target recovery interval that continuously meets the preset recovery degree threshold from this sequence, that is, the continuous delay range where the recovery degree is consistently high, rather than selecting a single peak point. This is because the compensation effect near the real acquisition delay will be at a relatively good level, and selecting a continuous interval can effectively eliminate delay misjudgment caused by a single noise point. Finally, based on the distribution characteristics of the recovery degree within the target recovery interval, the system selects the delay amount corresponding to the peak value of the recovery degree within the interval or the center value of the interval, and determines it as the final target acquisition delay amount. This target acquisition delay amount is the actual time lag value actually introduced by the image acquisition and key point recognition process, which is calculated.

[0028] S400: In response to the motion trajectory deviation meeting the feedback condition, output motion correction information through the display, obtain the corresponding correction response trajectory, and determine the target display delay based on the historical correction response baseline and the correction response trajectory.

[0029] Specifically, the calculated motion trajectory deviation is first used as the input to determine whether it meets the preset feedback conditions. These feedback conditions are pre-defined correction trigger rules, typically corresponding to situations where the spatial deviation of the motion trajectory exceeds a preset reasonable threshold or where the deviation occurs consecutively in the same motion phase and has room for correction. Setting these conditions avoids frequent feedback outputs that interfere with the training rhythm when the deviation is too small, and triggers subsequent processes only when the trainee actually needs motion guidance. When the motion trajectory deviation is determined to meet the feedback conditions, the system immediately responds and drives the display in the training scene to output motion correction information. The display is the visual output carrier for conveying correction guidance to the trainee, and the motion correction information is a concrete motion guidance content generated by combining the direction and magnitude of the deviation. It can be represented as a limb offset arrow, posture correction prompt, or standard posture comparison outline superimposed on the real-time training screen, used to clearly inform the trainee of the deviation position and adjustment direction of the current motion. The moment the system generates the correction information is the moment the system outputs the command. However, due to hardware limitations such as the display rendering time and screen refresh cycle, there will be a lag in the moment when the correction information is actually presented on the screen and perceived by the trainee. This lag is the core objective that needs to be quantified in this step.

[0030] After outputting the motion correction information, the system continuously captures the subsequent motion data of the training subject through front-end image acquisition and human key point tracking links. It extracts the complete motion trajectory segment from the moment the correction information system outputs the information to the moment the training subject receives the guidance and makes limb adjustment movements. This is the corresponding correction response trajectory. The trajectory completely records the key point position changes of the training subject from receiving the correction instruction to completing the motion correction. It includes the training subject's own neural reaction speed and motion adjustment process, and also superimposes the overall temporal offset caused by the lag in the display process. It is the core data basis for subsequent calculation of display latency.

[0031] The system then retrieves the historical correction response baseline that matches the current correction scenario. This historical correction response baseline is a benchmark response pattern obtained by statistically fitting multiple historical correction samples of the training object. The historical correction samples involved in the statistics all have a matching action stage and correction direction with the current action correction information. For example, in the knee adduction correction scenario, which belongs to the descent stage of a squatting action, this baseline has pre-eliminated the interference of display latency and only reflects the inherent reaction speed, action adjustment rate and response timing characteristics of the training object itself. It is a unified reference benchmark for measuring display latency and can avoid the measurement error caused by individual reaction differences in the general baseline.

[0032] Based on this, the system progressively calculates the target display latency: First, it determines multiple candidate display latency values ​​by combining the standard response time range of the historical correction response baseline with the monitor's refresh cycle. The monitor refresh cycle is the inherent time interval for the screen to complete one frame update, and it is also the smallest unit of display latency. All candidate display latency values ​​are integer multiples of the refresh cycle, representing multiple sets of display lag hypotheses to be verified. Then, for each candidate display latency value, the correction response trajectory undergoes a corresponding duration of temporal translation correction. The incremental motion of the target human body's key points along the target correction direction is extracted after correction, and this is used to form directional response features. The target correction direction is the direction indicated by the current correction information. Limb adjustment direction: Directional response features are used to quantify the amplitude and trend of limb movement adjustment along the direction of correction guidance. This can intuitively reflect whether the trainee has made corresponding directional adjustment movements in response to the correction command. Furthermore, based on the concentration of the motion increment within the candidate correction response period and its consistency with the target correction direction, the correction response coupling degree corresponding to the candidate delay is calculated. The correction response coupling degree is a quantitative indicator that measures the degree of matching between the correction response action after time-series correction and the historical baseline response pattern. The higher the coupling degree, the higher the degree of consistency between the trainee's response start time and adjustment rhythm and the historical baseline after the candidate delay is corrected, and the closer it is to the actual display lag time.

[0033] Finally, the system compares the correction response coupling degree corresponding to all candidate display delay values ​​and selects the candidate value corresponding to the peak value of the coupling degree as the target display delay value. This value is the quantified real display feedback lag time between the system generating the correction instruction and the training object actually perceiving the correction information.

[0034] S500: Monitor the source of deviation of the motion trajectory based on the target acquisition delay and the target display delay to obtain image acquisition delay deviation, display feedback delay deviation and human motion deviation.

[0035] Specifically, using the pre-calculated target acquisition delay and target display delay as correction parameters, the deviation of the overall motion trajectory obtained by the preceding matching is monitored for deviation sources in multiple dimensions. Deviation source monitoring refers to the complete process of decomposing and attributing the overall trajectory deviation, which is a mixture of system hardware latency and human execution error, according to the link link in which the deviation occurs. The goal is to distinguish between false deviations caused by inherent system latency and true deviations caused by non-standard execution of the training object's movements. First, the deviation components introduced by the image acquisition link are quantified. Based on the determined target acquisition delay, a time-series correction operation is performed on the original human keypoint trajectory. Here, time-series correction specifically refers to shifting the timestamps corresponding to all frames of the entire human keypoint trajectory forward by the duration corresponding to the target acquisition delay. This offsets the time lag effect brought about by the entire process of image acquisition, signal transmission, and keypoint recognition algorithm processing from the data level, aligning the time axis of the trajectory with the physical timeline of the actual occurrence of the training object's movements.

[0036] After completing the timing correction, the overall deviation amplitude between the original trajectory before correction and the standard motion trajectory, as well as the deviation amplitude between the corrected trajectory and the standard motion trajectory, are calculated. The difference between the two is the image acquisition delay deviation. This deviation component is entirely caused by the hardware sampling time and algorithm processing time at the acquisition end. It manifests as spatial position deviation and motion rhythm deviation caused by trajectory timing misalignment. It belongs to the inherent deviation of the system link and has no direct correlation with the motion execution quality of the training object.

[0037] Next, the system quantifies the deviation component introduced by the display feedback link. Based on the target display delay, it performs a corresponding timing correction operation on the previously acquired correction response trajectory. This means shifting the time axis of the correction response trajectory forward by the duration of the target display delay to offset the lag in the presentation of correction information caused by the display rendering and screen refresh cycle. This restores the actual starting moment when the training subject receives the correction instruction and makes limb adjustments. By comparing the change in deviation of the correction response trajectory relative to the standard adjustment path before and after timing correction, the display feedback delay deviation can be obtained. This deviation component originates from the hardware lag characteristics of the feedback output link, which causes the starting moment of the correction response to shift backward and the response rhythm to be out of sync. This is also an inherent deviation on the system side, and is not a deviation in the training subject's reaction speed or adjustment action itself.

[0038] After completing the timing correction of system delays at both the acquisition and display ends and completely eliminating timing interference introduced by system links, the corrected human key point trajectories are compared with the standard action trajectories corresponding to the target training actions in a refined manner, focusing on the spatial position and timing rhythm of each action phase. The remaining trajectory differences between the two are the human action deviations. This deviation component is the real deviation caused purely by subjective execution factors such as the training subject's own inadequate action execution, inaccurate posture control, and non-compliance with standard action rhythms after excluding all system link interferences. It can objectively reflect the actual action quality of the training subject.

[0039] Furthermore, in the method provided in this application embodiment, the human body key point trajectory is matched with the standard motion trajectory to obtain the motion trajectory deviation. The method includes: determining multiple motion timing feature nodes in the human body key point trajectory and the standard motion trajectory respectively; establishing a node correspondence relationship between corresponding motion timing feature nodes; and determining the motion trajectory deviation based on the temporal difference and spatial position difference between the motion timing feature nodes with the node correspondence relationship.

[0040] Specifically, the process first involves determining the action temporal feature nodes separately. This means that for the previously output human keypoint trajectory and the pre-calibrated target training action standard trajectory, multiple action temporal feature nodes are independently identified and extracted. These action temporal feature nodes refer to the set of key points corresponding to the instantaneous postures that represent a significant change in limb movement state and have clear action phase boundaries within the complete execution cycle of the action. Examples include the initial posture node at the moment the action starts, the limit posture node when the limb reaches its maximum range of motion, the reversal node when the direction of movement reverses, and the ending posture node when the action is completed and the body falls back down. Each feature node is bound to the precise timestamp of its occurrence and the spatial coordinates of all human keypoints. For the actual human keypoint trajectory, this determination operation locates the significant moment and extracts the corresponding node coordinates by calculating the extreme values ​​of the movement velocity, the acceleration passing zero point, and the extreme values ​​of the joint angles of each keypoint. For the standard action trajectory, the semantically similar phase feature nodes pre-calibrated by professional action calibration are directly retrieved to ensure that the feature nodes of the two types of trajectories have a natural correspondence in action phase semantics.

[0041] After extracting the feature nodes of the two types of trajectories, the operation of establishing the node correspondence between the corresponding action temporal feature nodes is performed. The node correspondence refers to the one-to-one pairing mapping relationship between the actual trajectory feature nodes and the standard trajectory feature nodes based on the same action phase semantics. This establishment operation is not a hard pairing according to the time sequence of node appearance, but rather an initial matching based on phase semantic type. Then, the duration ratio between adjacent feature nodes and the overall posture similarity are combined for verification and correction. For example, referring to the fixed proportion of the duration from the start to the peak phase in the standard trajectory to the total action duration, the duration ratio of the corresponding paired nodes in the actual trajectory is verified to see if it conforms to the basic logic of human action. This eliminates mismatches and ensures that each pair of paired nodes strictly corresponds to the same physical execution phase in the action execution process.

[0042] Based on this, the system calculates the temporal and spatial differences for each pair of feature nodes with corresponding relationships. The temporal difference refers to the difference between the timestamp of the feature node in the actual trajectory and the standard timestamp in the standard trajectory in the same phase. The positive and negative values ​​of the difference correspond to the lag and advance of the occurrence time of the action phase, respectively, and are used to quantify the deviation in the speed of action execution. The spatial difference refers to the difference in distance and angle between the spatial coordinates of the corresponding human key points in the actual trajectory and the standard coordinates in the standard trajectory in the same phase, and is used to quantify the degree of spatial deviation of the limb posture from the standard posture in the action phase. Subsequently, the system weights and summarizes the temporal differences of all paired feature nodes in the entire process according to the weight of the action stage to obtain the overall temporal deviation component. It also weights and merges the spatial differences of all corresponding key points according to the importance of the nodes to obtain the overall spatial deviation component. Finally, the two types of deviation components are normalized and integrated to obtain a comprehensive action trajectory deviation that simultaneously covers the temporal rhythm and spatial posture dimensions.

[0043] Furthermore, in the method provided in this application embodiment, multiple candidate acquisition delay amounts are determined according to the image acquisition cycle of the image sensor. The method further includes: determining an acquisition delay search interval according to the timing difference between corresponding action timing feature nodes; dividing the acquisition delay search interval according to the image acquisition cycle to obtain multiple candidate acquisition delay amounts.

[0044] Specifically, the system first determines the acquisition delay search interval based on the temporal differences between corresponding action temporal feature nodes. The temporal differences between corresponding action temporal feature nodes are the time difference between the occurrence of each pair of in-phase paired feature nodes obtained in the previous trajectory matching stage in the actual human keypoint trajectory and the standard action trajectory. This difference is mixed with the inherent lag of the image acquisition link and the fluctuation of the training object's own action rhythm, and cannot be directly equated to the real acquisition delay, but it can reflect the approximate magnitude range of the delay. The system uses the statistical median of the temporal differences of all paired feature nodes as the center of the interval, and expands the boundaries to both sides by combining the theoretical maximum time consumption of the entire link of image acquisition, transmission and keypoint recognition and the normal fluctuation range of human action rhythm. At the same time, the lower limit of the interval is constrained to a reasonable value not less than zero. Finally, the system defines the time range in which the real acquisition delay is likely to fall, which is the acquisition delay search interval. This interval can ensure that the real acquisition delay value is covered and avoid missed detection.

[0045] After defining the search interval, the system divides the acquisition delay search interval according to the image acquisition cycle of the image sensor. The image acquisition cycle is the inherent time interval for the image sensor to capture and output one frame of image, which is the smallest temporal granularity of the image sampling system. The actual image acquisition delay under the discrete sampling mechanism must be an integer multiple of the acquisition cycle. Therefore, the system takes the lower limit of the acquisition delay search interval as the starting point, and takes one image acquisition cycle as the fixed step size, and takes points in the direction of increasing time until the value covers the upper limit of the entire search interval. Each point value is a candidate acquisition delay value. All points together constitute multiple candidate acquisition delay values. Each candidate value is an integer multiple of the acquisition cycle, which perfectly matches the physical characteristics of discrete image sampling and ensures that the subsequent temporal compensation operation based on the candidate values ​​has real physical meaning.

[0046] Furthermore, in the method provided in this application embodiment, the target acquisition delay is determined based on the trajectory recovery characteristics of multiple target human key points after compensation. The method includes: for each candidate acquisition delay, determining the temporal recovery degree and relative geometric stability degree of multiple target human key points after temporal compensation; determining the corresponding acquisition recovery degree based on the temporal recovery degree and the relative geometric stability degree; and determining the target acquisition delay based on the acquisition recovery degree corresponding to the multiple candidate acquisition delays.

[0047] Specifically, the system first calculates the temporal recovery degree and relative geometric stability of multiple target human keypoints after temporal compensation for each candidate acquisition delay. The candidate acquisition delay is a set of acquisition lag assumptions obtained prior to the acquisition cycle division, whose authenticity needs to be verified. The target human keypoints are core skeletal nodes pre-selected from all human keypoints, exhibiting stable motion patterns in the target training movements and minimal interference from the trainee's subjective motion deformation. Typically, nodes with strong skeletal geometric constraints, such as the trunk center reference point, bilateral hip joints, and knee joints, are selected as spatial anchor points to verify the rationality of temporal compensation. Temporal compensation involves shifting the time axis of the entire human keypoint trajectory forward by the corresponding duration according to the current candidate acquisition delay, simulating the process of offsetting the acquisition link lag effect and restoring the actual moment of action from a data perspective. The temporal recovery degree is used to measure the performance after temporal compensation. The degree of temporal consistency between the action timing feature nodes of the actual human keypoint trajectory and the in-phase feature nodes of the standard action trajectory in the time dimension is specifically quantified by the contraction amplitude of the temporal difference of each pair of feature nodes before and after compensation. The higher the degree of temporal recovery, the better the match between the occurrence time of each phase of the actual action and the temporal rhythm of the standard action after the candidate delay amount is corrected. The relative geometric stability is used to measure the stability of the inherent relative geometric relationship of bones between multiple target keypoints between consecutive frames after temporal compensation. Based on the physical constraints of constant bone length and continuous and smooth joint movement in real human motion, if the temporal compensation amount does not match the actual acquisition delay, keypoints between frames will experience temporal misalignment or identity jumps, which will lead to abnormal fluctuations in the relative distance of bones and joint angles between adjacent frames. Therefore, the higher the relative geometric stability, the more the spatial shape of the compensated trajectory conforms to the objective physical laws of human motion.

[0048] After independently calculating the two indicators, the system performs a weighted fusion calculation of the temporal recovery degree and the relative geometric stability degree according to a preset weight ratio to obtain the acquisition recovery degree corresponding to the candidate acquisition delay. The acquisition recovery degree is a single quantitative evaluation index that integrates the temporal alignment effect and spatial geometric rationality. It can evaluate the offsetting effect of the candidate delay on the acquisition lag from both the temporal and physical constraint dimensions. The higher the value, the closer the candidate delay is to the real image acquisition delay. The dual-dimensional fusion judgment method can effectively avoid the defect of a single index being easily affected by spontaneous fluctuations in the rhythm of human movement, and greatly improve the anti-interference ability of the delay measurement results. After calculating the acquisition recovery degree corresponding to each of the candidate acquisition delays in sequence, the system performs an overall statistical analysis of all candidate values ​​and their recovery degree results, and selects the candidate value with the best acquisition recovery degree as the final target acquisition delay. Specifically, it can be determined by locating the global recovery degree peak point or by taking the center value in combination with continuous high recovery degree intervals, to ensure that the output delay can offset the temporal deviation caused by the acquisition link to the greatest extent.

[0049] Furthermore, in the method provided in this application embodiment, determining the target acquisition delay based on the acquisition recovery degree corresponding to the plurality of candidate acquisition delay amounts includes: forming an acquisition recovery degree sequence according to the magnitude of the candidate acquisition delay amounts; determining a target recovery interval that continuously satisfies the recovery degree condition from the acquisition recovery degree sequence; and determining the target acquisition delay based on the acquisition recovery degree distribution within the target recovery interval.

[0050] Specifically, the process first involves forming a sequence of acquisition recovery degrees based on the magnitude of the candidate acquisition delays. The candidate acquisition delays are a set of discrete acquisition lag hypothesis values ​​to be verified, generated in the acquisition delay search interval with the image acquisition period as the step size. Each candidate value has its own acquisition recovery degree calculated. The acquisition recovery degree is a two-dimensional compensation effect evaluation index that combines the temporal alignment effect and the rationality of the skeletal geometric constraints. The higher the value, the better the effect of the delay in offsetting the acquisition link lag.

[0051] The specific operation to form the sequence is to sort all candidate acquisition delays in ascending order according to their time values, and at the same time, arrange the acquisition recovery degree corresponding to each candidate value in a completely consistent order to form an ordered data sequence in which the delay and recovery degree are one-to-one. This sequence can intuitively show the overall trend of the acquisition recovery degree gradually increasing with the delay.

[0052] Next, the operation of determining the target recovery interval that continuously meets the recovery condition from the acquired recovery sequence is performed. The recovery condition is a pre-set minimum recovery threshold representing the acceptable level of temporal compensation. Only candidate delays with a recovery degree not lower than this threshold are considered to have a reliable compensation effect. When determining the interval, instead of directly selecting a single candidate point with the highest recovery degree, the process involves traversing and filtering point by point in the ordered recovery sequence. First, all candidate delays that meet the recovery condition are marked. Then, continuous and uninterrupted candidate value segments are extracted from all marked points. Each continuous qualified candidate value segment and the time range it covers constitute a recovery interval. The continuous segment with the best overall recovery performance and the largest number of candidate points covered is the target recovery interval. The core reason for selecting continuous intervals instead of single peaks is that in the vicinity of the actual image acquisition delay, the compensation effect of different candidate delays will remain at a high level. A single peak point may be caused by accidental factors such as inter-frame data fluctuations and key point recognition noise. Only continuous high recovery intervals can reflect the reasonable distribution range of the actual delay and effectively eliminate the interference of outliers.

[0053] The following operation determines the target acquisition delay based on the acquisition recovery distribution within the target recovery interval. The acquisition recovery distribution refers to the variation pattern and morphological characteristics of the recovery values ​​corresponding to each candidate delay within the target recovery interval. In the specific calculation, the distribution pattern of the recovery within the interval is first analyzed. If there is a clear single-peak distribution characteristic within the interval, that is, there is a clear point of highest recovery with values ​​smoothly decreasing on both sides, then the candidate acquisition delay corresponding to the peak point is directly selected as the final result. If the recovery within the interval is generally stable without significant fluctuations, then the midpoint value of the target recovery interval is used as the target acquisition delay. Alternatively, the weighted center value can be obtained by weighting each candidate value within the interval according to the recovery level as the final result. The final output target acquisition delay is the measured real temporal lag value introduced by the entire link of image acquisition and key point recognition.

[0054] Furthermore, in the method provided in this application embodiment, the historical correction response baseline is determined based on multiple historical correction samples of the training object, and the multiple historical correction samples have matching action stages and correction directions with the current action correction information.

[0055] Specifically, the historical correction response baseline is a benchmark response curve that is obtained through statistical fitting for a specific correction scenario for a specific training subject. After pre-extracting system display delay interference, it purely reflects the inherent neural reaction speed, limb adjustment rate and response timing pattern of the training subject. It includes core timing features such as standard response start delay, peak adjustment rate and response stability duration. It is a unified reference scale for measuring the degree of timing deviation of the actual correction response trajectory. Its construction data comes from multiple historical correction samples of the training subject.

[0056] Historical correction samples are complete correction response trajectory data segments collected and stored synchronously during the training process of the training object when it received movement correction information and made limb adjustment movements in previous training. Each sample is accompanied by annotations of the execution stage of the corresponding movement, the spatial direction of the correction guidance, and the full spatiotemporal data of the response trajectory. It is the original data material for fitting and generating the baseline.

[0057] Sample selection must meet the constraints of having a matching action stage and correction direction with the current action correction information. The action stage is an independent sub-stage divided into the target training action according to the execution sequence and movement state, such as the descent and standing stages of a squatting action, and the lifting and lowering stages of an upper limb pressing action. The force state, motion inertia, and difficulty of action adjustment of the human limbs are significantly different in different action stages, and the corresponding correction response laws are also different. The correction direction is the spatial direction of limb adjustment guided by the current action correction information, such as knee adduction, trunk straightening, and wrist return to center. Different adjustment directions involve different control muscle groups and movement patterns, and the human body's response speed and adjustment characteristics are also different.

[0058] Specifically, the system first identifies the action stage and correction direction of the action correction information to be output. Then, it searches and filters the historical sample database of the training object, extracting all historical correction samples with consistent action stage and correction direction identifiers. At the same time, invalid samples with abnormal situations such as external force interference or mid-action pauses are removed, resulting in multiple valid historical correction samples that match the scene. Then, the response trajectories of all valid samples are aligned to the zero point of time when the correction information is actually perceived by the training object. The display delay component superimposed on the historical data is removed in advance. Then, the multiple time-aligned response trajectories are statistically fitted. By calculating the statistical mean of the response amplitude at each time, extracting common response features and smoothing them, a statistically representative historical correction response baseline is finally formed.

[0059] Furthermore, in the method provided in this application embodiment, determining the target display latency based on the historical correction response baseline and the correction response trajectory includes: determining multiple candidate display latency amounts based on the historical correction response baseline and the refresh cycle of the display; for each candidate display latency amount, determining the directional response characteristics of the correction response trajectory along the target correction direction; determining the corresponding correction response coupling degree based on the response matching degree between the directional response characteristics and the historical correction response baseline; and determining the target display latency amount based on the correction response coupling degrees corresponding to the multiple candidate display latency amounts.

[0060] Specifically, the process first involves determining multiple candidate display latency values ​​based on the historical correction response baseline and the monitor's refresh cycle. The historical correction response baseline is a pre-fitted benchmark timing curve that purely reflects the training subject's own neural response and limb adjustment patterns. It has been stripped of system-side latency interference and includes core features such as the standard response start time and adjustment rate changes. The monitor's refresh cycle is the inherent time interval for the display screen to complete one frame update. It is the smallest granularity of display latency, and the actual display latency must be an integer multiple of this cycle.

[0061] In the specific determination, the standard response start delay of the historical correction response baseline and the output time of the system-side correction command are combined to define a reasonable search range for display delay. Then, the search range is uniformly and discretely divided with the display refresh cycle as a fixed step size. Points are taken sequentially from the lower limit to the upper limit of the range to obtain multiple candidate display delay values. Each candidate value is an integer multiple of the refresh cycle, representing a set of display lag duration assumptions to be verified. This ensures that the candidate values ​​are fully aligned with the physical characteristics of the display hardware and avoids meaningless non-full frame delay calculations.

[0062] Subsequently, for each candidate display delay, the system performs an operation to determine the directional response characteristics of the correction response trajectory along the target correction direction. The correction response trajectory is the complete key point motion trajectory of the training object after receiving the correction information and making limb adjustments. It is superimposed with the dual temporal offset of display lag and the human body's own reaction. The target correction direction is the spatial direction of limb adjustment clearly indicated by the correction information, corresponding to the correction direction of the current action deviation. This operation first shifts the time axis of the correction response trajectory forward by the corresponding time according to the current candidate display delay, completing the temporal pre-correction under the candidate assumption. Then, it extracts the motion data of the target human key points from the pre-corrected trajectory, projects its spatial motion vector onto the target correction direction, calculates the time-by-time motion increment and cumulative adjustment amplitude along this direction, and finally forms a directional response characteristic that only reflects the orientation adjustment process. This characteristic filters out motion interference from directions unrelated to the limb and can characterize the orientation response process of the training object to the correction command.

[0063] Based on this, the system performs an operation to determine the corresponding correction response coupling degree according to the degree of response matching between the directional response characteristics and the historical correction response baseline. The degree of response matching is the degree of consistency between the temporal pre-corrected directional response curve and the historical baseline in terms of temporal shape and change pattern. Specifically, it covers multiple dimensions such as the alignment of the response start time, the similarity of the adjustment rate, and the deviation of the response peak occurrence time. The correction response coupling degree is the quantitative output of the degree of matching. Its calculation combines the degree of concentration of motion increment in the candidate correction response period and the consistency of the target correction direction. That is, the more concentrated the adjustment action is in the standard response period corresponding to the baseline, and the higher the consistency between the motion direction and the correction guidance direction, the higher the coupling degree value. This means that the candidate delay amount has a better effect on offsetting display lag, and the corrected response pattern is more in line with the inherent response characteristics of the training object.

[0064] Finally, the system performs the operation of determining the target display delay based on the coupling degree of the correction response corresponding to multiple candidate display delays. That is, it iterates through all candidate display delays and their coupling degree results, and selects the candidate value with the peak coupling degree as the final target display delay. This value is the calculated real display feedback lag time from when the system generates the correction instruction to when the training object actually perceives the correction information.

[0065] Furthermore, in the method provided in this application embodiment, the directional response feature is determined based on the motion increment of the target human body key points along the target correction direction, and the correction response coupling degree is determined based on the concentration of the motion increment within the candidate correction response period and the degree of consistency with the target correction direction.

[0066] Specifically, the generation of directional response features is first clarified. Directional response features are temporal indicators specifically used to characterize the directional adjustment actions made by the training object in response to the correction command. The calculation anchors two core elements: the target human body key points and the target correction direction. The target human body key points are the core skeletal nodes directly corresponding to the current correction action. For example, the knee joint point is selected for the knee adduction correction scenario, and the thoracic reference point is selected for the torso straightening correction scenario. They are the direct bearing nodes of the limb adjustment action. The target correction direction is the limb correction spatial direction guided by the correction information of this action, corresponding to the reverse correction vector of the current action deviation.

[0067] Specifically, the spatial coordinates of key points of the target human body in each frame are first projected onto the unit vector of the target correction direction to obtain the projected position value of the key point in that direction in each frame. Then, the difference between the projected positions between adjacent frames is calculated, which is the motion increment along the target correction direction at the corresponding moment. The motion increment can intuitively reflect the instantaneous adjustment amplitude and adjustment direction of the limb along the correction direction at each moment. The motion increments in the whole response cycle are connected in time sequence to finally form a complete directional response feature. This feature filters out irrelevant interference signals such as lateral swaying of limbs and related movements of other parts through directional projection, and can focus on the directional adjustment process of the training object in response to the correction command.

[0068] Based on this, the correction response coupling degree is calculated. The correction response coupling degree is a comprehensive quantitative value that measures the degree of matching between the response trajectory after candidate display delay correction and the historical baseline. Its calculation relies on two core attributes of motion increment. The first is the concentration of motion increment within the candidate correction response period. The candidate correction response period is a standard time window determined based on the historical correction response baseline, in which the training object makes movement adjustments after receiving instructions normally. The concentration degree is specifically obtained by calculating the proportion of the sum of motion increment amplitude within the candidate period to the total motion increment of the entire response cycle. The higher the proportion, the more concentrated the limb adjustment movement is within the standard response period, indicating a higher degree of consistency between the response timing after candidate delay correction and the time rhythm of the baseline. The second is the consistency between motion increment and the target correction direction. This indicator is obtained by statistically analyzing the proportion of positive motion increment, that is, the proportion of motion increment amplitude along the target correction direction to the total motion increment amplitude. The higher the proportion, the better the fit between the limb adjustment direction and the correction guidance direction, which can effectively eliminate the interference of reverse compensation movements and irrelevant directional swaying on the response matching.

[0069] Finally, the degree of concentration and the degree of consistency are weighted and fused according to preset weights to obtain the correction response coupling degree of the corresponding candidate display delay. The higher the coupling degree value, the more ideal the cancellation effect of the candidate delay on display lag.

[0070] Furthermore, in the method provided in this application embodiment, separating the source of deviation of the motion trajectory deviation based on the target acquisition delay and the target display delay includes: performing time-series correction on the human key point trajectory based on the target acquisition delay, and determining the image acquisition delay deviation based on the change in trajectory deviation before and after time-series correction; performing time-series correction on the correction response trajectory based on the target display delay, and determining the display feedback delay deviation based on the change in trajectory deviation before and after time-series correction; and determining the human motion deviation based on the trajectory difference between the human key point trajectory after time-series correction and the standard motion trajectory.

[0071] Specifically, the deviation source separation refers to breaking down the comprehensive motion trajectory deviation, which originally involved a mixture of image acquisition timing lag, display feedback timing lag, and human execution errors, into three independent link links that generate the deviation. The specific amplitude of each type of deviation is quantified. The core objective is to distinguish between false deviations caused by system hardware and links and true deviations caused by incomplete execution of the training object's movements. The system first performs deviation stripping calculation at the acquisition end and performs time-series correction on the original human keypoint trajectory based on the determined target acquisition delay. The target acquisition delay is the actual timing lag duration introduced by the entire link of image acquisition, transmission, and keypoint recognition, obtained through multi-candidate verification and trajectory recovery degree screening. The timing correction specifically involves uniformly shifting the timestamps corresponding to all frames in the human keypoint trajectory forward by the duration of the target acquisition delay. This offsets the time lag effect of the acquisition link from the data level, aligning the trajectory's time axis with the physical time axis of the actual occurrence of the training object's movements.

[0072] After the correction is completed, the overall deviation amplitude between the original human key point trajectory before time-series correction and the standard motion trajectory, and the deviation amplitude between the trajectory after time-series correction and the standard motion trajectory are calculated respectively. The difference between the two is the image acquisition delay deviation. This deviation component is entirely caused by the hardware sampling and algorithm processing time at the acquisition end. It manifests as spatial position deviation and motion rhythm deviation caused by time-series misalignment. It belongs to the inherent deviation of the system side and has no direct correlation with the motion execution quality of the training object.

[0073] Next, the deviation of the display feedback end is calculated. The timing of the correction response trajectory is corrected according to the target display delay. The target display delay is the actual display lag time from the generation of the correction command by the system to the actual perception of the correction information by the training object, which is obtained by matching the individual response baseline. The correction response trajectory is the complete key point movement trajectory of the training object after receiving the correction instruction and making limb adjustment movements. Its original timing is superimposed with the lag effect of the display link. The timing correction here is to shift the time axis of the correction response trajectory forward by the target display delay time, restore the starting time when the training object actually receives the correction command and starts the adjustment movement. Then, the deviation amplitude of the correction response trajectory before and after the timing correction relative to the standard adjustment path is calculated. The difference between the two is the display feedback delay deviation. This deviation component comes from the hardware characteristics of the display end such as screen rendering and screen refresh, which will cause the overall timing of the correction response to be delayed and the rhythm to be misaligned. It is also an inherent deviation on the system side, and is not a defect in the reaction speed of the training object or the adjustment movement itself.

[0074] After completing the timing correction of the latency of both the acquisition and display systems and completely eliminating timing interference introduced by all system links, the timing-corrected human key point trajectory is compared with the standard movement trajectory corresponding to the target training movement in a fine-grained phase-by-phase comparison. The remaining differences between the two in spatial posture, movement path and timing rhythm are calculated. This remaining difference is the human movement deviation. This deviation component is the real deviation caused purely by subjective execution factors such as insufficient movement control ability of the training object, inadequate posture mastery, and non-compliance with standard movement rhythm after eliminating all system interference. It can objectively and accurately reflect the actual movement quality of the training object.

[0075] In summary, the training motion trajectory deviation monitoring method based on human keypoint tracking provided in this application has the following technical effects: By acquiring training image sequences of the training subjects performing the target training actions and tracking the trajectory of human key points, the overall action trajectory deviation is obtained by matching these with the standard action trajectory. Then, multiple sets of candidate acquisition delays are set with the acquisition cycle of the image sensor as the smallest granularity. The target acquisition delay is determined iteratively by combining the trajectory recovery features of multiple target key points after time-compensation. At the same time, in response to the deviation meeting the feedback conditions, action correction information is output and the correction response trajectory is acquired. The target display delay is calculated by combining the historical correction response baseline that matches the current action stage and correction direction. Finally, the deviation sources of the overall action trajectory deviation are separated and monitored based on the two types of delays, and three independent components are separated: image acquisition delay deviation, display feedback delay deviation, and human action deviation. Without the need for additional hardware acquisition equipment, the interference of the inherent delay of the system link on the action deviation judgment is effectively eliminated, improving the accuracy of action trajectory deviation assessment and the fineness of deviation source location.

[0076] Example 2 is based on the same inventive concept as the training motion trajectory deviation monitoring method based on human key point tracking in the previous examples, such as... Figure 2 As shown in the embodiment of this application, a training motion trajectory deviation monitoring device based on human keypoint tracking is provided. The device includes: The tracking module 11 is used to acquire the training image sequence corresponding to the target training action performed by the training object, and to track human key points in the training image sequence to obtain the human key point trajectory; the matching module 12 is used to match the human key point trajectory with the standard action trajectory corresponding to the target training action to obtain the action trajectory deviation; the compensation module 13 is used to determine multiple candidate acquisition delays based on the image acquisition cycle of the image sensor, and to perform time-series compensation on the human key point trajectory according to each candidate acquisition delay, and to determine the target acquisition delay based on the trajectory recovery characteristics of multiple target human key points after compensation; the correction module 14 is used to output action correction information through the display in response to the action trajectory deviation meeting the feedback condition, to acquire the corresponding correction response trajectory, and to determine the target display delay based on the historical correction response baseline and the correction response trajectory; the monitoring module 15 is used to monitor the source of the action trajectory deviation based on the target acquisition delay and the target display delay, and to obtain the image acquisition delay deviation, the display feedback delay deviation, and the human action deviation.

[0077] Furthermore, the matching module 12 is also used to perform the following steps: determine multiple action timing feature nodes in the human body key point trajectory and the standard action trajectory respectively; establish node correspondence between corresponding action timing feature nodes; and determine the action trajectory deviation based on the timing difference and spatial position difference between the action timing feature nodes with the node correspondence.

[0078] Furthermore, the compensation module 13 is also used to perform the following steps: determine the acquisition delay search interval based on the timing difference between the corresponding action timing feature nodes; divide the acquisition delay search interval according to the image acquisition cycle to obtain multiple candidate acquisition delay amounts.

[0079] Furthermore, the compensation module 13 is also used to perform the following steps: for each of the candidate acquisition delay amounts, determine the temporal recovery degree and relative geometric stability degree of multiple target human body key points after temporal compensation; determine the corresponding acquisition recovery degree according to the temporal recovery degree and the relative geometric stability degree; determine the target acquisition delay amount according to the acquisition recovery degree corresponding to the multiple candidate acquisition delay amounts.

[0080] Furthermore, the compensation module 13 is also used to perform the following steps: forming a collection recovery degree sequence according to the magnitude of the candidate collection delay; determining a target recovery interval that continuously meets the recovery degree condition from the collection recovery degree sequence; and determining the target collection delay amount according to the collection recovery degree distribution within the target recovery interval.

[0081] Furthermore, the correction module 14 is also used to perform the following steps: the historical correction response baseline is determined based on multiple historical correction samples of the training object, and the multiple historical correction samples have matching action stages and correction directions with the current action correction information.

[0082] Furthermore, the correction module 14 is also used to perform the following steps: determining multiple candidate display delay amounts based on the historical correction response baseline and the refresh cycle of the display; determining the directional response characteristics of the correction response trajectory along the target correction direction for each candidate display delay amount; determining the corresponding correction response coupling degree based on the response matching degree between the directional response characteristics and the historical correction response baseline; and determining the target display delay amount based on the correction response coupling degree corresponding to the multiple candidate display delay amounts.

[0083] Furthermore, the correction module 14 is also used to perform the following steps: the direction response feature is determined based on the motion increment of the target human body key points along the target correction direction, and the correction response coupling degree is determined based on the concentration of the motion increment in the candidate correction response period and the consistency with the target correction direction.

[0084] Furthermore, the correction module 14 is also used to perform the following steps: perform time-series correction on the trajectory of the human body key points according to the target acquisition delay amount, and determine the image acquisition delay deviation according to the change in trajectory deviation before and after time-series correction; perform time-series correction on the correction response trajectory according to the target display delay amount, and determine the display feedback delay deviation according to the change in trajectory deviation before and after time-series correction; determine the human body movement deviation according to the trajectory difference between the human body key point trajectory after time-series correction and the standard motion trajectory.

[0085] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for monitoring training motion trajectory deviation based on human keypoint tracking, characterized in that, The methods include: Obtain the training image sequence corresponding to the target training action performed by the training object, and perform human key point tracking on the training image sequence to obtain the human key point trajectory; The trajectory of the human body key points is matched with the standard movement trajectory corresponding to the target training movement to obtain the movement trajectory deviation; Multiple candidate acquisition delays are determined based on the image acquisition cycle of the image sensor. The trajectory of the human body key points is compensated for time sequence according to each of the candidate acquisition delays. The target acquisition delay is determined based on the trajectory recovery features of the multiple target human body key points after compensation. In response to the motion trajectory deviation satisfying the feedback condition, motion correction information is output through the display to obtain the corresponding correction response trajectory, and the target display delay is determined based on the historical correction response baseline and the correction response trajectory. Based on the target acquisition delay and the target display delay, the source of the motion trajectory deviation is monitored to obtain the image acquisition delay deviation, display feedback delay deviation, and human motion deviation.

2. The method for monitoring training motion trajectory deviation based on human keypoint tracking as described in claim 1, characterized in that, The method of matching the trajectory of the human body key points with the standard motion trajectory to obtain the motion trajectory deviation includes: Determine multiple action temporal feature nodes in the human body key point trajectory and the standard action trajectory, respectively; Establish the node correspondence between corresponding action sequence feature nodes; The action trajectory deviation is determined based on the temporal and spatial differences between action timing feature nodes with the aforementioned node correspondence.

3. The method for monitoring training motion trajectory deviation based on human keypoint tracking as described in claim 2, characterized in that, The method for determining multiple candidate acquisition delays based on the image acquisition period of the image sensor also includes: The acquisition delay search interval is determined based on the temporal differences between corresponding action temporal feature nodes; The acquisition delay search interval is divided according to the image acquisition period to obtain multiple candidate acquisition delay amounts.

4. The method for monitoring training motion trajectory deviation based on human keypoint tracking as described in claim 1, characterized in that, The target acquisition delay is determined based on the trajectory recovery features of multiple target human body key points after compensation. Methods include: For each of the candidate acquisition delays, determine the temporal recovery degree and relative geometric stability of multiple target human key points after temporal compensation; The corresponding acquisition recovery degree is determined based on the temporal recovery degree and the relative geometric stability degree; The target acquisition delay is determined based on the acquisition recovery rate corresponding to the multiple candidate acquisition delays.

5. The method for monitoring training motion trajectory deviation based on human keypoint tracking as described in claim 4, characterized in that, Determining the target acquisition delay based on the acquisition recovery rates corresponding to multiple candidate acquisition delays includes: A sequence of acquisition recovery degrees is formed according to the magnitude of the candidate acquisition delay. Determine the target recovery interval that continuously satisfies the recovery condition from the acquired recovery sequence; The target acquisition delay is determined based on the acquisition recovery distribution within the target recovery interval.

6. The method for monitoring training motion trajectory deviation based on human keypoint tracking as described in claim 1, characterized in that, The historical correction response baseline is determined based on multiple historical correction samples of the training object, and the multiple historical correction samples have matching action stages and correction directions with the current action correction information.

7. The method for monitoring training motion trajectory deviation based on human keypoint tracking as described in claim 1, characterized in that, Determining the target display delay based on the historical correction response baseline and the correction response trajectory includes: Multiple candidate display latency values ​​are determined based on the historical correction response baseline and the refresh cycle of the display; For each of the candidate display delay amounts, determine the directional response characteristics of the correction response trajectory along the target correction direction; The corresponding correction response coupling degree is determined based on the degree of response matching between the directional response characteristics and the historical correction response baseline; The target display delay is determined based on the correction response coupling degree corresponding to the multiple candidate display delays.

8. The method for monitoring training motion trajectory deviation based on human keypoint tracking as described in claim 7, characterized in that, The directional response feature is determined based on the motion increment of key points of the target human body along the target correction direction, and the correction response coupling degree is determined based on the concentration of the motion increment during the candidate correction response period and the degree of consistency with the target correction direction.

9. The method for monitoring training motion trajectory deviation based on human keypoint tracking as described in claim 1, characterized in that, Based on the target acquisition delay and the target display delay, the source of the motion trajectory deviation is separated, including: The trajectory of the human body key points is time-corrected based on the target acquisition delay, and the image acquisition delay deviation is determined based on the change in trajectory deviation before and after the time-correction. The timing correction of the correction response trajectory is performed based on the target display delay amount, and the display feedback delay deviation is determined based on the change in trajectory deviation before and after the timing correction. The human motion deviation is determined based on the trajectory difference between the human key point trajectory after time-correction and the standard motion trajectory.

10. A training motion trajectory deviation monitoring device based on human key point tracking, characterized in that, The device is used for implementing the training motion trajectory deviation monitoring method based on human keypoint tracking according to any one of claims 1 to 9, wherein the device comprises: The tracking module is used to acquire the training image sequence corresponding to the training object performing the target training action, and to perform human key point tracking on the training image sequence to obtain the human key point trajectory. The matching module is used to match the trajectory of the human body key points with the standard motion trajectory corresponding to the target training motion to obtain the motion trajectory deviation; The compensation module is used to determine multiple candidate acquisition delays based on the image acquisition cycle of the image sensor, perform time-series compensation on the trajectory of the human body key points according to each of the candidate acquisition delays, and determine the target acquisition delay based on the trajectory recovery features of the multiple target human body key points after compensation. The correction module is used to respond to the fact that the deviation of the motion trajectory meets the feedback condition, output motion correction information through the display, obtain the corresponding correction response trajectory, and determine the target display delay based on the historical correction response baseline and the correction response trajectory. The monitoring module is used to monitor the source of deviation of the motion trajectory based on the target acquisition delay and the target display delay, and to obtain the image acquisition delay deviation, display feedback delay deviation and human motion deviation.

Images