A training effect analysis method and system of motion video and wearable data linkage

Abstract

The application discloses a kind of training effect analysis method and system of motion video and wear data linkage, belong to motion fitness and intelligent analysis technical field, solve the pain point that existing motion analysis can only output independent score, cannot distinguish motion deformation inducement;The scheme synchronously collects motion video and wearable device multi-source data, completes time alignment and clock drift correction by multidimensional synchronous credibility verification;Parallelly carry out motion quality analysis and physiological load evaluation, correlate data based on unified time axis, realize abnormal attribution by cause-effect coupling score and rule base, can distinguish different inducements such as motion non-standard or physiological fatigue;Solve the problem that motion and physiological data are difficult to linkage analysis in prior art, feedback is rough, can accurately identify the root of fatigue, technical deformation, excessive intensity and other problems in training, and provide real-time, individualized multi-level feedback, thereby significantly improve the scientificity, safety and user experience of training analysis.

Description

Technical Field

[0001] This invention relates to the field of motion analysis technology, specifically a method and system for analyzing training effects by linking motion videos with wearable data. Background Technology

[0002] With the increasing popularity of home fitness and smart training devices, users increasingly expect to receive simultaneous assessments of movement quality, physiological load, and personalized training suggestions during their workouts. Current technologies typically suffer from the following shortcomings: First, the training guidance methods are relatively simple. Some systems only rely on video images for motion recognition or only rely on wristbands for heart rate monitoring. It is difficult to link and analyze the motion performance with the physiological state, so it is impossible to accurately determine whether the motion deformation is caused by fatigue, excessive intensity or lack of skill.

[0003] Second, there is a lack of reliable synchronization mechanism between video data and wearable data. During training, there are often discrepancies between the camera and mobile phone system time and the wearable device time. Without a unified timeline and synchronization calibration mechanism, data from different sources are difficult to correspond accurately, affecting the credibility of the analysis results.

[0004] Third, the existing feedback methods are rather crude. Most training applications only provide simple voice reminders, action scores, or statistics on the number of times completed. They cannot output emergency feedback, important feedback, and prompt feedback according to the degree of abnormality, nor can they form a closed loop of "identifying problems - analyzing causes - giving suggestions - recording events - post-training review" during the training process.

[0005] Therefore, there is a need for a training effect analysis method that can uniformly synchronize, analyze in real time, perform causal reasoning, and provide multi-level feedback on motion video data and wearable device data, in order to improve the recognition accuracy, feedback timeliness, and personalization during the training process. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a method and system for analyzing training effects by linking motion videos and wearable data, which can effectively solve the problems mentioned in the background art.

[0007] To address the above problems, the technical solution adopted by this invention is: a method for analyzing training effects by linking motion videos with wearable data, comprising the following steps: S1. Synchronously collect user's motion video data and wearable device data; S2. Perform time alignment processing on the motion video data and wearable device data to establish a data mapping relationship under a unified time axis; S3. Based on the time-aligned data, the causal relationship between athletic performance and physiological indicators is analyzed by a causal reasoning engine to generate training effect analysis results. The causal reasoning engine performs multi-factor fusion reasoning based on a preset rule base. S4. Based on the training effect analysis results, generate and provide feedback information; In addition, the training effect analysis results are also used to simultaneously output movement quality, physiological load, causal attribution results, and feedback information during the training process.

[0008] Preferably, before step S1, there is a pre-training initialization step, which includes at least device connection and environment detection, acquisition of user basic information, establishment of personal exercise baseline and synchronous start-up. The synchronous startup includes at least one of audio synchronization signal, visual synchronization signal and system timestamp synchronization, and the real-time processing loop in training is only entered when the synchronization reliability meets a preset threshold. The synchronization reliability is calculated according to the following formula: ; In the formula, To ensure synchronization reliability; Score for audio synchronization; Score for visual synchronization; The timestamp consistency score; The score is the clock drift correction score; λ1, λ2, λ3, and λ4 are weighting coefficients, and λ1+λ2+λ3+λ4=1.

[0009] Preferably, the time alignment process in step S2 includes at least one of the following methods: a. Synchronization alignment based on audio signals, which is performed by detecting preset synchronization audio events; b. Visual-action-based synchronization alignment, which performs alignment by recognizing preset synchronization actions; c. Synchronization alignment based on system timestamps; It also registers the video timestamp with the wearable device timestamp according to the time mapping table, and performs drift correction when clock drift is detected.

[0010] Preferably, step S3 includes: S31. Perform real-time analysis on the motion video data to obtain motion analysis results including motion type, motion quality score, and motion anomaly markers; S32. Perform real-time analysis on the data from the wearable device to obtain physiological state analysis results including heart rate zones, exercise intensity levels, and fatigue index; S33. Correlate the action analysis results with the physiological state analysis results on a unified time axis; S34. Call the rule base in the causal reasoning engine to perform reasoning. The rule base contains multiple preset rules for judging the causal relationship between athletic performance and physiological state, and outputs training effect analysis results including attribution results, confidence level and suggested measures. The quality score for the movement is calculated using the following formula: ; In the formula: Rate the quality of the movement; For the symmetry component of the action; For the amplitude component of the movement; For the consistency component of motion speed; is the attitude stability component; a1, a2, a3, and a4 are weighting coefficients, and a1+a2+a3+a4=1.

[0011] Preferably, the wearable device data includes at least one of heart rate data, exercise intensity data, cadence data, and calorie consumption data from the fitness tracker; the physiological state analysis results include at least heart rate change trends, exercise intensity levels, and fatigue index. The fatigue index is calculated according to the following formula: ; In the formula: Current heart rate; This is the resting heart rate; Maximum heart rate; The fatigue index; Physiological stress score is calculated using the following formula: ; In the formula: Assess physiological workload; For heart rate reserve related indicators; This represents the rate of change in heart rate over the last 30 seconds. , represents the calorie consumption related items; b1, b2, b3, and b4 are weighting coefficients, and b1+b2+b3+b4=1.

[0012] Preferably, the causal reasoning engine performs reasoning based on the following causal coupling score: ; In the formula, Scoring for causal coupling; For Sigmoid mapping functions; This represents the time lag between changes in movement and changes in heart rate; c1 represents the matching degree of historical behavior patterns; c1, c2, c3, and c4 are weight coefficients, and c1+c2+c3+c4=1; The training risk value is calculated using the following formula: ; In the formula, To train risk values; β1, β2, and β3 are the synchronization reliability; β1, β2, and β3 are the weighting coefficients, and β1 + β2 + β3 = 1.

[0013] The training risk value and / or causal coupling score are used to generate at least one of voice feedback, AR overlay display feedback, and vibration feedback.

[0014] Preferably, step S4 includes real-time feedback and post-training feedback; The real-time feedback includes at least voice guidance information, augmented reality information overlaid on the exercise video, and vibration alerts triggered by the wearable device; the post-training feedback includes a training overview, causal diagnostic conclusions, timeline analysis, personalized suggestions, and progress tracking.

[0015] Preferably, it also includes a key event recording step and an individual baseline update step; When a sudden drop in motion quality, abnormal heart rate fluctuations, a causal coupling score exceeding a preset threshold, or a user-initiated marker is detected, a critical event record is generated. The key event record shall include at least the event timestamp, event type, video clip, attribution results, confidence level, recommended actions, and data comparison before and after the event; The critical event score is calculated according to the following formula: ; In the formula, Score for key events; This refers to the sudden change in motion quality. This refers to the amount of abnormal fluctuations in heart rate. These are user-initiated tags; d1, d2, d3, and d4 are weight coefficients, and d1+d2+d3+d4=1.

[0016] The individual baseline update is calculated according to the following formula: ; In the formula, For the user's individual baseline at time t; These are the effective feature vectors extracted during this training process; This is the historical memory coefficient.

[0017] A training effect analysis system that links motion video and wearable data to implement the above analysis method includes: a data acquisition module for synchronously acquiring user motion video data and wearable device data; The data processing and alignment module is used to perform time alignment processing on the motion video data and the wearable device data to establish a data mapping relationship under a unified time axis. The causal analysis and reasoning module is used to analyze the causal relationship between athletic performance and physiological indicators based on time-aligned data through a causal reasoning engine, and generate training effect analysis results. The causal reasoning engine performs multi-factor fusion reasoning based on a preset rule base. The feedback output module is used to generate and provide feedback information based on the training effect analysis results; The causal inference engine may employ: rule-based inference models; Bayesian network models; dynamic Bayesian network models; structured causal models; time series causal analysis models; or combinations of the above models.

[0018] A computer-readable storage medium having a computer program stored thereon, the computer program implementing the method when executed by a processor. Compared with existing technologies, the present invention provides a method and system for analyzing training effects by linking motion videos with wearable data, which has the following beneficial effects: This invention improves the accuracy and reliability of training data correlation analysis by synchronizing and jointly analyzing motion video data and wearable device data.

[0019] By integrating reasoning based on movement quality, physiological load, and causal coupling, problems such as fatigue, movement distortion, and abnormal training intensity can be accurately identified, thereby improving the ability to analyze training effects.

[0020] By using multi-level feedback methods such as voice, AR, and vibration, real-time intervention and personalized guidance can be provided during the training process, thereby improving training safety and user experience.

[0021] By using key event recording, historical data analysis, and individual baseline update mechanisms, we can track training effectiveness and optimize long-term training, thereby improving personalized analysis capabilities.

[0022] Through fault tolerance and rollback mechanisms, basic analysis capabilities can still be maintained even when video is obscured or wearable devices are disconnected, thus improving system stability and usability. Attached Figure Description

[0023] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a system architecture block diagram of the present invention. Detailed Implementation

[0024] The present invention will now be described with reference to embodiments. It should be understood that the embodiments described herein are for the purpose of more clearly illustrating the invention, and not for limiting the scope of the invention.

[0025] The endpoints and any values ​​of the ranges disclosed herein are not limited to the precise ranges or values, and these ranges or values ​​should be understood to include values ​​close to these ranges or values. For numerical ranges, the endpoint values ​​of the various ranges, the endpoint values ​​of the various ranges and individual point values, and individual point values ​​can be combined with each other to obtain one or more new numerical ranges, which should be considered as specifically disclosed herein.

[0026] refer to Figure 1-2 As a specific embodiment 1 of the present invention: Normal determination in a standard squat training scenario: In this embodiment, the user performs squat training in the living room at home, with the mobile phone fixed on a stand and the wristband connected to the mobile phone via Bluetooth.

[0027] User Operation: 1. Open the App → Select the exercise type (e.g., "Squat Training") 2. Fix the phone position (stand recommended / automatic detection of shooting angle) 3. Connect the wristband (Bluetooth automatic / manual) 4. Perform the calibration exercise (3 standard squats) System processing: Detects lighting conditions (recommended >200 lux), detects shooting range (whether the whole body can be seen), obtains basic user information (age, gender, height, weight), and establishes personal motion baseline (based on historical data or a general model); The ambient light was 260 lux, which met the shooting conditions recommended in the solution. The user first completed three standard squat calibrations and then triggered the audio synchronization signal by clapping three times. The system recorded the correspondence between the video timestamp and the wristband timestamp.

[0028] In this embodiment, the synchronization reliability formula is taken as: ; During the initial training phase, the system measured: =0.97, =0.95, =0.99, =0.94; Substituting, we get: =0.35×0.97+0.25×0.95+0.25×0.99+0.15×0.94=0.9655; That is, the synchronization reliability is 96.55%, meeting the conditions for formal analysis.

[0029] During the real-time analysis phase, the video-side motion quality score is recorded as follows: ; During this squat, the system detected: =0.88; =0.86; =0.84; =0.90; Substituting, we get: =0.30×0.88+0.30×0.86+0.20×0.84+0.20×0.90=0.870; That is, the quality of the movement is 87.0 points.

[0030] The physiological load score on the wearing side is recorded as follows: ; During this training session, the wristband measured: =0.52; =0.50; =0.48; =0.55; Substituting, we get: =0.35×0.52+0.25×0.50+0.25×0.48+0.15×0.55=0.5095; That is, the physiological load is 50.95 points.

[0031] In the causal reasoning stage, this embodiment takes: ; In this training: 1− =0.13; =0.40; =0.85; Substituting, we get: ; That is, the causal coupling score is 61.6%.

[0032] Based on the above results, the system determines that the movement quality is high, the physiological load is within an acceptable range, and the causal coupling relationship does not constitute an anomaly. Therefore, the system outputs: Voice prompt: "Very good! Your current posture is standard, keep up the pace." AR display: Motion score 87 / 100, heart rate 128 bpm, condition is normal; The bracelet only generates a slight, alert vibration and does not trigger emergency feedback; The post-training report concluded that "the training was generally stable."

[0033] As a specific embodiment 2 of the present invention: fatigue accumulation in the later stages of training leads to movement deformation: In this embodiment, the user performed 45 minutes of squat training. The movement was relatively stable for the first 20 minutes, but gradually the knees buckled inward and the lower back bent over in the last 15 minutes. This situation is consistent with the rule in the instruction plan that "when the quality of movement decreases by more than 20% and the heart rate increases by more than 20%, it is attributed to fatigue leading to movement deformation."

[0034] In the later stages of training, the system detected that: the video-based motion quality score dropped from 86 to 64, a decrease of 25.6%; the heart rate measured by the wristband increased from 140 bpm to 168 bpm, an increase of 20%; the fatigue index increased from 62% to 84%; and the movement rhythm slowed down significantly in the last 30 seconds, with an increased slope of heart rate change.

[0035] In real-time computing, on the video side: =0.642, meaning the quality of the movement is 64.2 points; Wearable side: =0.7965, which means the physiological load is 79.65 points.

[0036] In the causal coupling stage, take: 1− =0.358; =0.75; =0.78; but: That is, the causal coupling score is 66.7%; The system further generates key event scores: ; This time, no manual marking by the user is required. =0.

[0037] The system detected: =0.256, =0.20, =0.667; After the weights are set, If the critical event threshold is exceeded, the system determines that the segment is "fatigue accumulation leading to motion deformation".

[0038] The system output is as follows: Voice feedback: "Fatigue detected. We suggest you rest for 30 seconds before continuing."

[0039] AR feedback: The action score changes from green to orange; a red circle appears on the knee; the screen displays "Suggestion: Reduce movement speed and pay attention to posture".

[0040] Vibration feedback: The bracelet triggers continuous vibration, which is an important alert level.

[0041] Event Log: Cause: Accumulated fatigue; Confidence level: Approximately 0.85; Recommendation: Extend rest time between sets to 60 seconds.

[0042] Post-training report: The report marked this period as a "weak period," indicating that the performance score dropped by 25% in the last 15 minutes, while the heart rate increased by 20%, which is a typical fatigue pattern.

[0043] As a specific embodiment 3 of the present invention: the movement is basically normal but the heart rate is abnormally high, which is judged to be too intense. In this embodiment, the user performs high-intensity squat training with generally good range of motion and symmetry, but the heart rate continues to rise to more than 90% of the maximum heart rate, which is consistent with the rule in the briefing plan that "when the quality of the movement is normal but the heart rate is abnormally high, it is attributed to excessive intensity leading to physiological stress".

[0044] The system detected: Action quality score =0.84, or 84 points; 30-second heart rate variability rate significantly increased; current heart rate reached 182 bpm, accounting for 93% of the expected maximum heart rate; fatigue index rose above 90%; wearable device calculations showed: =0.953; that is, the physiological load is 95.3 points.

[0045] The video-side motion quality is high, therefore: 1− =0.16; Historical pattern matching degree =0.76, timing lag measurement =0.65, then: The causal coupling score was 66.3%.

[0046] At this point, the system does not consider the problem to be with the movement technique, but instead judges it based on the rule base as "excessive exercise intensity leading to physiological stress," and outputs: Voice prompt: "Current intensity is too high, please reduce training load immediately." AR display: Heart rate value turns red, higher than the recommended range; The wristband triggered a strong vibration alert; Training suggestions: Reduce movement speed, extend rest between sets, and shorten the number of sets in this round of training; The training report marks the event as an "intensity problem mode" and prompts the user to adjust the weight or number of repetitions in subsequent training.

[0047] As a specific embodiment 4 of the present invention: fault-tolerant rollback processing during video interruption: In this embodiment, a brief video occlusion occurs during user training, but the wristband continues to upload heart rate and acceleration data. The disclosed solution has revealed a fault-tolerant mechanism that "uses wristband data to estimate actions when the video is interrupted, uses video data to estimate heart rate when the wristband disconnects, and uses the last valid data plus a time decay model in the event of a double interruption."

[0048] When the video is interrupted for 8 seconds, the system executes a rollback strategy: taking the last 5 seconds of valid action status as a reference; using heart rate changes and acceleration fluctuations on the wristband side to infer the current action rhythm; reducing the weight of the video side and increasing the weight of the wearer side; and outputting a low-confidence prompt instead of directly making a strong attribution.

[0049] In this embodiment, the system outputs the following results: the confidence level of causal inference drops to 0.48; the system prompts: "Insufficient data, please ensure clear shooting and correct wristband wearing."; the current event is marked as "event to be confirmed"; no explicit attribution is output to avoid misjudgment.

[0050] This embodiment illustrates that the system has fault tolerance and degradation processing capabilities in abnormal scenarios, which can ensure uninterrupted analysis and avoid erroneous feedback caused by missing data on one side.

[0051] The technical solutions of the various embodiments can be combined with each other, but must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

[0052] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural transformations made using the contents of the present invention under the inventive concept of the present invention, or direct / indirect applications in other related technical fields, are included within the patent protection scope of the present invention.

Claims

1. A method for analyzing training effects by linking motion videos with wearable data, characterized in that, Includes the following steps: S1. Synchronously collect user's motion video data and wearable device data; S2. Perform time alignment processing on the motion video data and wearable device data to establish a data mapping relationship under a unified time axis; S3. Based on the time-aligned data, the causal relationship between athletic performance and physiological indicators is analyzed by a causal reasoning engine to generate training effect analysis results. The causal reasoning engine performs multi-factor fusion reasoning based on a preset rule base. S4. Based on the training effect analysis results, generate and provide feedback information; In addition, the training effect analysis results are also used to simultaneously output movement quality, physiological load, causal attribution results, and feedback information during the training process.

2. The method for analyzing training effects by linking motion video and wearable data according to claim 1, characterized in that, Before step S1, there is a pre-training initialization step, which includes at least device connection and environment detection, acquisition of basic user information, establishment of personal exercise baseline and synchronous start-up. The synchronous startup includes at least one of audio synchronization signal, visual synchronization signal and system timestamp synchronization, and the real-time processing loop in training is only entered when the synchronization reliability meets a preset threshold. The synchronization reliability is calculated according to the following formula: ； In the formula, To ensure synchronization reliability; Score for audio synchronization; Score for visual synchronization; The timestamp consistency score; The score is the clock drift correction score; λ1, λ2, λ3, and λ4 are weighting coefficients, and λ1+λ2+λ3+λ4=1.

3. The method for analyzing training effects by linking motion video and wearable data according to claim 1, characterized in that, The time alignment process in step S2 includes at least one of the following methods: a. Synchronization alignment based on audio signals, which is performed by detecting preset synchronization audio events; b. Visual-action-based synchronization alignment, which performs alignment by recognizing preset synchronization actions; c. Synchronization alignment based on system timestamps; It also registers the video timestamp with the wearable device timestamp according to the time mapping table, and performs drift correction when clock drift is detected.

4. The method for analyzing training effects by linking motion video and wearable data according to claim 1, characterized in that, Step S3 includes: S31. Perform real-time analysis on the motion video data to obtain motion analysis results including motion type, motion quality score, and motion anomaly markers; S32. Perform real-time analysis on the data from the wearable device to obtain physiological state analysis results including heart rate zones, exercise intensity levels, and fatigue index; S33. Correlate the action analysis results with the physiological state analysis results on a unified time axis; S34. Call the rule base in the causal reasoning engine to perform reasoning. The rule base contains multiple preset rules for judging the causal relationship between athletic performance and physiological state, and outputs training effect analysis results including attribution results, confidence level and suggested measures.

5. The method for analyzing training effects by linking motion video and wearable data according to claim 4, characterized in that, The wearable device data includes at least one of heart rate data, exercise intensity data, cadence data, and calorie consumption data from the fitness tracker; the physiological state analysis results include at least heart rate change trends, exercise intensity levels, and fatigue index. The fatigue index is calculated according to the following formula: ； In the formula: Current heart rate; This refers to the resting heart rate. Maximum heart rate; This represents the fatigue index.

6. The method for analyzing training effects by linking motion video and wearable data according to claim 1, characterized in that, The causal reasoning engine performs inferences based on the following causal coupling score: ； In the formula, Scoring for causal coupling; For Sigmoid mapping functions; This refers to the time lag between changes in movement and changes in heart rate; represents the historical behavior pattern matching degree; c1, c2, c3, and c4 are weight coefficients, and c1+c2+c3+c4=1.

7. The method for analyzing training effects by linking motion video and wearable data according to claim 1, characterized in that, Step S4 includes real-time feedback and post-training feedback; The real-time feedback includes at least voice guidance information, augmented reality information overlaid on the exercise video, and vibration alerts triggered by the wearable device; the post-training feedback includes a training overview, causal diagnostic conclusions, timeline analysis, personalized suggestions, and progress tracking.

8. The training effect analysis method based on the linkage of motion video and wearable data according to claim 1 further includes a key event recording step and an individual baseline update step; When a sudden drop in motion quality, abnormal heart rate fluctuations, a causal coupling score exceeding a preset threshold, or a user-initiated marker is detected, a critical event record is generated. The key event record shall include at least the event timestamp, event type, video clip, attribution results, confidence level, recommended actions, and data comparison before and after the event; The critical event score is calculated according to the following formula: ； In the formula, Score for key events; This refers to the sudden change in motion quality. This refers to the amount of abnormal fluctuations in heart rate. These are user-initiated tags; d1, d2, d3, and d4 are weight coefficients, and d1+d2+d3+d4=1; The individual baseline update is calculated according to the following formula: ； In the formula, For the user's individual baseline at time t; These are the effective feature vectors extracted during this training process; This is the historical memory coefficient.

9. A training effect analysis system that links motion videos with wearable data, used to implement the training effect analysis method of any one of claims 1-8, characterized in that, include: The data acquisition module is used to simultaneously collect the user's motion video data and wearable device data; The data processing and alignment module is used to perform time alignment processing on the motion video data and the wearable device data to establish a data mapping relationship under a unified time axis. The causal analysis and reasoning module is used to analyze the causal relationship between athletic performance and physiological indicators based on time-aligned data through a causal reasoning engine, and generate training effect analysis results. The causal reasoning engine performs multi-factor fusion reasoning based on a preset rule base. The feedback output module is used to generate and provide feedback information based on the training effect analysis results.

Images