Intelligent correction method and system for physical action of primary and secondary school students based on multi-modal perception

Abstract

The present application relates to the technical field of sports action teaching, and particularly relates to a middle and primary school student sports action intelligent correction method and system based on multi-modal perception; the present application realizes from student action data input to root cause joint accurate positioning and then to targeted correction output through a complete technical chain of multi-modal data acquisition and fusion, standard timing template comparison, directed graph deviation backtracking propagation and root cause joint identification and correction, overcomes the defect that the prior art can only detect single joint deviation and cannot trace the root cause, and significantly improves the efficiency of sports action correction; the present application dynamically adjusts the fusion weight of each mode in real time through visual detection confidence and inertial signal signal-to-noise ratio, effectively overcomes the defect that single visual acquisition is easily disturbed by light change and body shielding, and greatly improves the accuracy and stability of joint angle data in complex teaching environment, providing a high-quality data basis for subsequent root cause positioning.

Description

Technical Field

[0001] This invention relates to the field of physical education teaching technology, and in particular to an intelligent error correction method and system for physical education movements of primary and secondary school students based on multimodal perception. Background Technology

[0002] In daily physical education teaching in primary and secondary schools, teachers mainly rely on on-site guidance to correct students' incorrect movements. However, due to the current large class sizes, teachers find it difficult to observe and guide each student's subtle limb movements, force application timing, joint angles, and other details during training, and can only provide rough corrections for obvious errors. With the development of AI technology, there are now error correction systems that collect videos and match training movements with standard demonstration movements frame by frame or time sequence. These systems detect deviations in parameters such as joint angles and positions to determine whether the movement is correct and provide correction prompts.

[0003] However, existing error correction systems can only detect whether a deviation occurred at a specific joint at a particular moment, but they struggle to trace the root cause of the problem. In athletic movements, there are strict temporal and mechanical coupling relationships between joints during force exertion. For example, the error of "low release angle" in shot put may not be rooted in the wrist or elbow joint itself, but rather in "insufficient hip joint force during the push-off phase," leading to inefficient kinetic chain transmission. This causes subsequent joints to deviate under compensatory action. Existing error correction systems often only assess each joint independently, lacking the ability to model and analyze the error transmission path between joints. When a downstream joint deviates, the system can only provide information such as "elbow angle too large" or "wrist trajectory deviation," making it difficult to trace the deviation back in the reverse direction of force transmission to determine if it originated from an upstream joint's error transmission. This makes it difficult for students to identify the core cause affecting movement quality, resulting in low error correction efficiency. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides an intelligent error correction method and system for primary and secondary school students' physical education movements based on multimodal perception. This solves the technical problem that current error correction systems for correcting students' physical education movements are unable to trace the core causes of errors in students' movements along the reverse direction of force transmission, resulting in low error correction efficiency.

[0005] To address the aforementioned technical problems, this invention provides the following technical solution: a method for intelligent error correction of physical education movements in primary and secondary school students based on multimodal perception, comprising the following steps: S1. Multimodal data acquisition and preprocessing.

[0006] First, data on multiple modalities of students performing the target physical activity are collected. In this invention, this includes at least acquiring video stream data of students performing the target physical activity using a visual camera. Then, the two-dimensional coordinates of key points of the human skeleton in each frame of the video stream data are extracted. The three-dimensional spatial position of each joint is obtained through triangulation reconstruction, and the real-time angle value of each joint is calculated. The real-time angle value of the j-th joint in frame t can be expressed as: Simultaneously, it also includes acquiring real-time acceleration and angular velocity signals of each joint through wearable inertial sensors worn on the student's limbs, obtaining sensor data, and then calculating the angle values ​​based on this data. Then, the angle values ​​calculated from both video stream data and sensor data are timestamped, filtered, and denoised, and then fused using a weighted fusion algorithm to output the fused angle value of each joint in each frame. The fused angle value of the j-th joint in frame t can then be expressed as... The total number of frames is In this invention, the weighted fusion algorithm employs an adaptive weight allocation mechanism based on signal confidence, which specifically includes the following steps: First, obtain the angle values ​​corresponding to the video stream data and sensor data, respectively. Let the angle value of the j-th joint calculated from the video stream data in frame t be... Let the angle value of the j-th joint in frame t be calculated from sensor data. .

[0007] Then, the dynamic correction amount for any frame is calculated based on the confidence level of the video stream acquisition device and the sensor signal quality. The calculation formula is as follows: In the above formula, The dynamic correction amount for frame t is determined by both the confidence level of the video stream acquisition device and the sensor signal quality for frame t. The confidence score of the visual human keypoint detection algorithm in frame t is obtained by normalizing the peak response intensity of the heatmap of the skeletal keypoint extraction algorithm. The closer the value is to 1, the more reliable the visual detection result. This represents the signal-to-noise ratio of the sensor signal in frame t, and can be output in real time by the sensor's built-in analog-to-digital converter. The threshold is a preset visual confidence level. If the video stream data is below this threshold, it is determined that the video stream data is significantly affected by occlusion or lighting interference. The preset sensor signal-to-noise ratio threshold is used. If the signal-to-noise ratio is lower than this threshold, the sensor signal is considered to be significantly affected by motion artifacts or poor contact. This is a preset adjustment factor used to control the sensitivity of the weights to changes in signal quality. The larger the value, the steeper the weight switching. The default value is 2. This is the hyperbolic tangent function, used to smoothly constrain the values ​​within parentheses to the interval (-1, 1). Next, set constraints for the weighting, and calculate the weighting of the video stream data and sensor data based on the dynamic correction amount and the preset base weights of the sensor data. The calculation formula is as follows: and The following constraints must be satisfied: In the above formula, and These are the weighted weights for video stream data and sensor data, respectively. and They are respectively and The corresponding base weights, and The sum is 1. and They are respectively and The corresponding weighting coefficients, and These are the preset minimum limits for visual weights and sensor weights, respectively. , These are the preset minimum limits for visual weights and sensor weights, respectively, to prevent a single signal source from being completely discarded under extreme conditions, which would lead to distortion of the fused data.

[0008] Finally, the angle values ​​corresponding to the video stream data and sensor data are weighted and fused according to their respective weights, and constraints are applied to obtain the fused angle value. The weighted fused angle value of the j-th joint in frame t is then calculated. for: In the above formula, and They are respectively and The corresponding weighting coefficients, and The sum is 1.

[0009] S2, Standard motion dataset matching and instantaneous deviation value calculation.

[0010] Based on the type identifier of the current target sports movement, the standard timing template of the corresponding movement is retrieved from the pre-constructed dataset of standard sports movements for primary and secondary school students. This standard timing template contains the standard angle values ​​of each joint in each frame. Therefore, the standard angle value of the j-th joint in frame t can be expressed as... Then, the fused angle values ​​are subtracted from the standard angle values ​​frame by frame and joint by joint to calculate the instantaneous deviation value of each joint. The instantaneous deviation value of the j-th joint in frame t can then be expressed as: Its calculation formula can be expressed as: S3. Root cause joint localization based on deviation backtracking propagation and directed graph model.

[0011] First, a directed graph model of the human motion joint chain corresponding to the target sports movement is pre-constructed. , where the set of nodes The set of directed edges corresponding to the N joints involved in this action. According to the definition of the direction of force transmission between human joints, for a set of directed edges Each directed edge in Associated with the transmitted frame delay parameter and force transmission coefficient In this invention, the transmission frame delay parameter and force transmission coefficient When determining the parameters, the joint kinematic parameters of the standard human body model in the pre-constructed dataset of standard physical education movements for primary and secondary school students are first used as a benchmark. The proportional relationship between the height and limb length of the standard human body model and the student to be tested is calculated, and the transmission frame delay parameter is adjusted accordingly. Personalized adjustments are then made. Specifically, the transmission frame delay parameter is first calculated. The benchmark value It is extracted from the standard motion dataset according to the following formula: in, and When the target sports movement is performed by a standard human body model, the upstream node and downstream nodes The frame number in which the peak angular velocity was reached. It is a positive integer; The personalized conduction frame delay parameters of the student under test Calculate using the following formula: In the above formula, To start from the upstream node to downstream nodes The limb length proportionality factor is calculated using the following formula: in, Indicates the upstream node of the student to be tested. to downstream nodes The actual length of the limb segments between the corresponding joints. Represents the upstream node of the standard human body model to downstream nodes The actual length of the limb segment between the corresponding joints.

[0012] Then, regarding the force transmission coefficient For personalized adjustments, the first step is to calculate the force transmission coefficient. The benchmark value It can be extracted from the standard motion dataset according to the following calculation formula: in, and When the target sports movement is performed by a standard human body model, the upstream node and downstream nodes The peak angular velocity value is then used to determine the individualized force transmission coefficient of the student being tested. Based on its benchmark value Taking into account the relative deviation between the ratio of the actual peak angular velocity of the student under test and the standard reference value, as well as the correction for the student's body mass index, the following formula is used for calculation: in, The deviation coefficient of the ratio of the actual peak angular velocity of the student under test to the standard reference value is calculated as follows: In the above formula, and These are the upstream nodes when the student being tested performs the target physical activity. and downstream nodes The actual peak angular velocity value, This is a body mass index correction factor used to compensate for the deviation in force transmission efficiency caused by differences in physical development between the student being tested and a standard human model. Its calculation method is as follows: In the above formula, Body Mass Index (BMI) for a standard human model. The actual body mass index (BMI) of the student being tested can be calculated by dividing the body mass by the square of the height.

[0013] Then, based on the instantaneous deviation values ​​of each joint calculated in step S2... Deviation backpropagation is performed in the reverse direction of the directed graph model of the human motion joint chain, for each directed edge. Calculate the upstream node For downstream nodes Expected deviation contribution value in frame t The calculation formula is as follows: In the above formula, This is an indicator function that takes the value 1 when the condition inside the parentheses is true, and 0 otherwise.

[0014] Then, for each node Calculate its root cause confidence score. The calculation formula is as follows: in, Represents a node The maximum instantaneous deviation value during the entire target athletic movement. Indicates the basic weighting coefficient. Represents a node The set of direct downstream nodes, Indicates the backtracking attenuation coefficient. Indicates the total number of frames. This indicates the location weight of the downstream node.

[0015] Finally, the root cause confidence score was calculated. The node with the largest value is determined to be the root cause joint. .

[0016] In this step, the root cause joint is determined. In addition, a dynamic threshold determination process can be further performed, the specific steps of which are as follows: Let the set of root cause confidence scores for all joint nodes be denoted as . Sort them in descending order to obtain the sorted score sequence. ,in, and The maximum and second largest root cause confidence scores are given respectively. The proportion of the difference between the maximum and second largest root cause confidence scores is calculated to obtain the proportion coefficient, which is calculated using the following formula: in, , is the difference ratio coefficient. The closer this value is to 1, the greater the difference between the confidence scores of the largest root cause and the second largest root cause, and the higher the uniqueness of the root cause determination. The closer this value is to 0, the smaller the difference between the confidence scores of the largest root cause and the second largest root cause, and the more potential root cause joints exist. At this point, a preset dynamic threshold parameter is used. The difference ratio coefficient With dynamic threshold parameters Comparison: like Determine the maximum root cause confidence score. The corresponding joint node is the only root cause joint; like If multiple root cause joints are determined to exist, then the following conditions will be met. The joint nodes corresponding to the multiple root cause confidence scores are all regarded as root cause joints.

[0017] In this invention, the dynamic threshold parameter It can be further adaptively set according to the type of the current target sports movement, and the calculation formula is as follows: in, The preset base threshold can be set to 0.15. This is the movement type adjustment coefficient, determined by the joint coupling degree of the current target sports movement. Its value ranges from (0.5, 2.0). Joint coupling degree is a measure of the correlation between kinematic parameters between joints when performing the movement. For movements with higher coupling degrees, such as shot put and standing long jump (multi-joint coordinated explosive power movements), its... The larger the value, the more dynamic threshold parameter... The more relaxed the coupling, the more likely it is to output multiple root joints. For movements with lower coupling, such as high knees in place and single-joint isolation exercises, its... The smaller the value, the smaller the dynamic threshold parameter. The stricter the criteria, the more likely it is to output a single root cause.

[0018] In addition, the motion frame intervals caused by joint deviations were calculated separately for each root. ,in and These are the starting and ending frames where the instantaneous deviation value of the joint continuously exceeds a preset deviation threshold.

[0019] S4, Root Cause Joint Error Correction Matching and Error Correction Output.

[0020] According to the root cause joint and the action frame interval where the deviation occurs The invention retrieves a built-in sports movement professional error correction knowledge base. This knowledge base contains multiple error correction records, each associated with at least three core attribute fields: root cause joint tag. Action phase tags and the corresponding set of error correction content The error correction content set includes at least correction guidance text. Demonstration Action Diagram Labels and voice command encoding The error correction information in three modalities is used to correct the students' target physical movements.

[0021] Each target athletic movement is pre-divided into multiple movement phases, such as the "preparation phase," "power generation phase," "follow-up phase," and "recovery phase." Then, the root joints are calculated. Action frame interval where deviation occurs Median frame number Then query the action phase segmentation boundary set of the target sports action. ,in This refers to the boundary frame number of each action phase. The total number of action phases, when Then determine the root cause joint at this time. The action frame interval where the deviation occurred belongs to the first... Each action phase.

[0022] Then you can determine the root cause of the joint. The system searches and matches the corresponding action stages in the sports movement error correction knowledge base to find the corresponding error correction content set, and obtains the corresponding correction guidance text, demonstration action diagram, and voice command code. Then, the correction guidance text is superimposed on the corresponding joint position of the original video screen, and the demonstration action diagram is simultaneously superimposed on the original video screen to play the corresponding demonstration action synchronously. At the same time, the voice command code is broadcast to output voice commands for guidance, thus outputting a complete error correction prompt.

[0023] The present invention also provides a sports movement intelligent error correction system, including a processor and a memory, wherein the memory is used to store a computer program, and the computer program, when executed by the processor, implements the sports movement intelligent error correction method.

[0024] By employing the above technical solution, the present invention provides a method and system for intelligent error correction of physical education movements of primary and secondary school students based on multimodal perception, which has at least the following beneficial effects: 1. This invention achieves a complete technical chain from student movement data input to precise localization of root cause joints and then to targeted error correction output through multimodal data acquisition and fusion, standard time-series template comparison, directed graph deviation backtracking propagation, and root cause joint identification and correction. It overcomes the shortcomings of existing technologies that can only detect single joint deviations and cannot trace the root cause, and significantly improves the efficiency of sports movement error correction.

[0025] 2. This invention dynamically adjusts the fusion weights of each modality in real time by using visual detection confidence and inertial signal signal-to-noise ratio. This effectively overcomes the shortcomings of single visual acquisition, which is susceptible to changes in lighting and interference from limb occlusion. It significantly improves the accuracy and stability of joint angle data in complex teaching environments, providing a high-quality data foundation for subsequent root cause localization.

[0026] 3. When constructing a directed graph model of human motion joint chains, this invention introduces a limb length ratio factor and a body mass index correction factor for the students under test to make personalized corrections to the transmission frame delay parameter and force transmission coefficient, so that the error propagation model can adapt to students of different heights and weights, significantly improving the cross-age and cross-body generalization ability of root cause localization.

[0027] 4. This invention introduces a transmission frame delay parameter in the deviation backtracking propagation to ensure that the deviation of the downstream joint is caused only by the force state of the upstream joint at a sufficiently early moment. Furthermore, it integrates the maximum deviation of the joint itself with the total contribution to the downstream joint in the root cause confidence score. This achieves the technical effect of accurately separating the root cause from the manifestation in multi-joint coupled motion data. The output root cause localization conclusion has clear interpretability and biomechanical rationality.

[0028] 5. This invention introduces a comparison mechanism between the difference ratio coefficient and the dynamic threshold in the root cause determination stage. When the root cause confidence scores of multiple joints are not significantly different, multiple candidate root cause joints are automatically output, which effectively addresses the actual situation of multiple joint coordination errors occurring concurrently in complex sports movements and significantly improves the practicality and pertinence of error correction suggestions. Attached Figure Description

[0029] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of the intelligent error correction method for physical education movements of primary and secondary school students based on multimodal perception, according to the present invention. Detailed Implementation

[0030] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. This will allow for a full understanding of how the present application uses technical means to solve technical problems and achieve technical effects, and to facilitate its implementation.

[0031] To address the technical problem of low error correction efficiency caused by the inability of current error correction systems for student physical education movements to trace the core causes of errors along the reverse direction of force transmission, this embodiment provides an intelligent error correction method for primary and secondary school students' physical education movements based on multimodal perception. Taking the teaching scenario of shot put throwing in primary and secondary schools as an example, the method is explained in detail. Shot put throwing is a typical multi-joint coordinated force-generating movement, with force transmitted from the lower limbs through the trunk to the upper limbs, involving the sequential coordination of multiple joints such as the ankle, knee, hip, shoulder, elbow, and wrist. The method specifically includes the following steps: First, a high-definition USB camera was installed on both the front and side of the throwing area, with a frame rate of 30fps and a resolution of 1920×1080, to simultaneously capture dual-view video streams of the student's shot put throwing motion. Simultaneously, six-axis inertial sensors were attached to four key locations on the student's right wrist, hip, right knee, and right ankle, with a sampling frequency of 100Hz. Data was wirelessly transmitted to a host computer via Bluetooth 5.0. A standard teaching computer equipped with an Intel Core i5 processor and 8GB of RAM was used as the host computer to run this method. Before starting, the height (h) and mass (m) of the student were obtained through conventional anthropometry, and their body mass index (BMI) was calculated. The BMI of the standard anthropomorphic model was provided by a standard motion dataset.

[0032] S1. Multimodal data acquisition and preprocessing.

[0033] When a student performs a shot put throw, video stream data of the student's action is captured by a visual camera. This video stream data is then input into a human skeletal keypoint detection algorithm (such as OpenPose) to extract the two-dimensional coordinates of key points in each frame. Using the parallax information from both perspectives, the three-dimensional spatial coordinates of each joint are obtained through triangulation reconstruction, including but not limited to: left / right ankle, left / right knee, left / right hip, left / right shoulder, left / right elbow, and left / right wrist. The real-time angle values ​​of each joint are then calculated. The real-time angle value of the j-th joint in frame t can be expressed as: Simultaneously, six-axis inertial sensors worn at each joint acquire three-axis acceleration and angular velocity signals in real time at a frequency of 100Hz, obtaining sensor data. After receiving the sensor data, the host computer first performs zero-bias correction and low-pass filtering (cutoff frequency 10Hz), then uses an attitude calculation algorithm (such as the Madgwick algorithm) to fuse the acceleration and angular velocity data into quaternions, and then converts the quaternions into Euler angles to obtain the angle values ​​of each joint. Then, the angle values ​​calculated from the video stream data and sensor data are timestamped, filtered, and denoised, and then a weighted fusion algorithm is used to fuse the two modalities, outputting the fused angle value of each joint in each frame. Therefore, the fused angle value of the j-th joint in frame t can be expressed as... The total number of frames is In this embodiment, the weighted fusion algorithm employs an adaptive weight allocation mechanism based on signal confidence, which specifically includes the following steps: Let the angle value of the j-th joint in frame t be calculated from the video stream data. Let the angle value of the j-th joint in frame t be calculated from sensor data. Then the angle value of the j-th joint after weighted fusion in the t-th frame. for: In the above formula, and They are respectively and The corresponding weighting coefficients, and The sum of the two is 1, and they can be calculated using the following formula: in, and They are respectively and The corresponding base weights, and The sum is 1. The dynamic correction amount for frame t is determined by both the confidence level of the video stream acquisition device and the sensor signal quality for frame t. Its calculation formula is as follows: In the above formula, The confidence score of the visual human keypoint detection algorithm in frame t is obtained by normalizing the peak response intensity of the heatmap of the skeletal keypoint extraction algorithm. The closer the value is to 1, the more reliable the visual detection result. This represents the signal-to-noise ratio of the sensor signal in frame t, and can be output in real time by the sensor's built-in analog-to-digital converter. The threshold is a preset visual confidence level. If the video stream data is below this threshold, it is determined that the video stream data is significantly affected by occlusion or lighting interference. The preset sensor signal-to-noise ratio threshold is used. If the signal-to-noise ratio is lower than this threshold, the sensor signal is considered to be significantly affected by motion artifacts or poor contact. This is a preset adjustment factor used to control the sensitivity of the weights to changes in signal quality. The larger the value, the steeper the weight switching. The default value is 2. This is the hyperbolic tangent function, used to smoothly constrain the values ​​within parentheses to the interval (-1, 1). Additionally, it can also be used for... Set the following constraints: in, , These are the preset minimum limits for visual weights and sensor weights, respectively, to prevent the complete discarding of a single signal source under extreme conditions, which could lead to distortion of the fused data. For example, when... Below and Higher than This indicates that the visual data is affected by occlusion or lighting interference. Towards The direction is lowered; when Below ,and Higher than This indicates that the inertial signal is interfered with by motion artifacts. Towards The direction is upward.

[0034] S2, Standard motion dataset matching and instantaneous deviation value calculation.

[0035] First, a pre-constructed dataset of standard physical education movements for primary and secondary school students is built, which includes a standard timing template for the shot put throwing motion. This template is based on data collected from standard human models of primary and secondary school students of the same age and gender performing the standard shot put throwing motion, and includes the standard angle values ​​of each joint in each frame. Then, based on the type identifier of the current target physical education movement, the corresponding standard timing template for the shot put throwing motion is retrieved from the pre-constructed dataset of standard physical education movements for primary and secondary school students. This template contains the standard angle values ​​of each joint in each frame. Therefore, the standard angle value of the j-th joint in frame t can be represented as... Then, the fused angle value is compared with the standard angle value frame by frame and joint by joint to calculate the instantaneous deviation value of each joint. The instantaneous deviation value of the j-th joint in frame t can then be expressed as: Its calculation formula can be expressed as: The instantaneous deviation value of each joint in frame t is calculated sequentially using this formula. For the shot put throwing action, this embodiment pays special attention to the instantaneous deviation values ​​of six joints: right ankle, right knee, right hip, right shoulder, right elbow, and right wrist.

[0036] S3. Root cause joint localization based on deviation backtracking propagation and directed graph model.

[0037] First, a directed graph model of the human motion joint chain corresponding to the target sports movement is pre-constructed. , where the set of nodes Corresponding to the six main joints involved in this movement, including the right ankle, right knee, right hip, right shoulder, right elbow, and right wrist, then... ={ =Right ankle, =right knee, =Right hip, =Right shoulder, = right elbow, =right wrist}, set of directed edges Based on the definition of the direction of force transmission between human joints, in this embodiment, the force direction is mainly from the lower limbs to the trunk and then to the distal part of the upper limbs, for example... ={ =(Right ankle → Right knee), =(Right knee → Right hip), =(Right hip → Right shoulder), =(Right shoulder → Right elbow), =(right elbow → right wrist)}. For the set of directed edges. Each directed edge in Associated with the transmission frame delay parameter and force transmission coefficient In this embodiment, the transmission frame delay parameter and force transmission coefficient When determining the parameters, the joint kinematic parameters of the standard human body model in the pre-constructed dataset of standard physical education movements for primary and secondary school students are first used as a benchmark. The proportional relationship between the height and limb length of the standard human body model and the student to be tested is calculated, and the transmission frame delay parameter is adjusted accordingly. Personalized adjustments are made. Specifically, the frame sequence number at which each joint reaches its peak angular velocity during a shot put throw using a standard human model is used as a benchmark to calculate the transmitted frame delay parameter. The benchmark value It is extracted from the standard motion dataset according to the following formula: in, and When the target sports movement is performed by a standard human body model, the upstream node and downstream nodes The frame number in which the peak angular velocity was reached. It is a positive integer; Based on benchmark value Then the personalized transmission frame delay parameter of the student to be tested Calculate using the following formula: In the above formula, For upstream nodes to downstream nodes The limb length proportionality factor is calculated using the following formula: in, Indicates the upstream node of the student to be tested. to downstream nodes The actual length of the limb segments between the corresponding joints. Represents the upstream node of the standard human body model to downstream nodes The actual length of the limb segment between the corresponding joints.

[0038] Taking the femoral segment between the right knee and right hip as an example: If the femoral length in a standard human model is 0.38m, and the femoral length of the student being tested is 0.40m, then... =0.40 / 0.38≈1.05. =7 frames, then Frame, thus through The corrections made the actual It is designed to fit the student's actual body size.

[0039] Then, regarding the force transmission coefficient For personalized adjustments, the first step is to calculate the force transmission coefficient. The benchmark value It can be extracted from the standard motion dataset according to the following calculation formula: in, and When the target sports movement is performed by a standard human body model, the upstream node and downstream nodes The peak angular velocity value is then used to determine the individualized force transmission coefficient of the student being tested. Based on its benchmark value Taking into account the relative deviation between the ratio of the actual peak angular velocity of the student being tested and the standard reference value, as well as the correction for the student's body mass index, the following formula is used for calculation: in, The deviation coefficient of the ratio of the actual peak angular velocity of the student under test to the standard reference value is calculated as follows: In the above formula, and These are the upstream nodes when the student to be tested performs the target physical activity. and downstream nodes The actual peak angular velocity value, This is a body mass index correction factor used to compensate for the deviation in force transmission efficiency caused by differences in physical development between the student being tested and a standard human model. Its calculation method is as follows: In the above formula, Body Mass Index (BMI) for a standard human model. The actual body mass index (BMI) of the student being tested can be calculated by dividing the body mass by the square of the height. This indicates that the conduction coefficient needs to be appropriately increased to compensate for the loss of conduction efficiency that may be caused by insufficient muscle mass.

[0040] Then, based on the instantaneous deviation values ​​of each joint calculated in step S2... Deviation backpropagation is performed in the reverse direction of the directed graph model of the human motion joint chain, for each directed edge. Calculate the upstream node For downstream nodes Expected deviation contribution value in frame t The calculation formula is as follows: In the above formula, This is an indicator function that takes the value 1 when the condition inside the parentheses is true, and 0 otherwise.

[0041] Then, for each node Calculate its root cause confidence score. The calculation formula is as follows: in, Represents a node The maximum instantaneous deviation value during the entire target athletic movement. This represents the base weighting coefficient. In this embodiment, the base weighting coefficient for the right ankle is 1, and the base weighting coefficients for the other joints are 0.5. Represents a node The set of direct downstream nodes; in the chain structure of this embodiment, each node has only one direct downstream node. This represents the backflashdown coefficient, which can be set to 0.7. Indicates the total number of frames. The positioning weights of downstream nodes are determined by the biomechanical importance of each joint in the shot put throwing motion, with relatively high weights for the hip and shoulder joints, and relatively low weights for the ankle and wrist joints. All joints have relatively low weights. The sum is 1. The root cause probability of a node is determined by its own maximum deviation and the total deviation contribution to all downstream joints, and nodes closer to the starting point of force are given higher basic weights.

[0042] Finally, in this embodiment, a dynamic threshold determination process is performed, which is combined with the root cause confidence score. Value determination of root cause joint The specific steps are as follows: Let the set of root cause confidence scores for all joint nodes be denoted as . Sort them in descending order to obtain the sorted score sequence. ,in, and The maximum and second largest root cause confidence scores are given respectively. The proportion of the difference between the maximum and second largest root cause confidence scores is calculated to obtain the proportion coefficient, which is calculated using the following formula: in, , is the difference ratio coefficient. The closer this value is to 1, the greater the difference between the confidence scores of the largest root cause and the second largest root cause, and the higher the uniqueness of the root cause determination. The closer this value is to 0, the smaller the difference between the confidence scores of the largest root cause and the second largest root cause, and the more potential root cause joints exist. At this point, a preset dynamic threshold parameter is used. The difference ratio coefficient With dynamic threshold parameters Comparison: like Determine the maximum root cause confidence score. The corresponding joint node is the only root cause joint; like If multiple root cause joints are determined to exist, then the following conditions will be met. The joint nodes corresponding to the multiple root cause confidence scores are all regarded as root cause joints.

[0043] In this embodiment, the dynamic threshold parameter It can be further adaptively set according to the type of the current target sports movement, and the calculation formula is as follows: in, The preset base threshold can be set to 0.15. This is the movement type adjustment coefficient, determined by the joint coupling degree of the current target sports movement. Its value ranges from (0.5, 2.0). Joint coupling degree is a measure of the correlation between kinematic parameters between joints when performing the movement. For movements with higher coupling degrees, such as shot put and standing long jump (multi-joint coordinated explosive power movements), its... The larger the value, the better. Generally, a value between 1.0 and 2.0 is used for the dynamic threshold parameter. The more relaxed the coupling, the more likely it is to output multiple root joints. For movements with lower coupling, such as high knees in place and single-joint isolation exercises, its... The smaller the value, typically between 0.5 and 1.0, the better the dynamic threshold parameter. The stricter the criteria, the more likely it is to output a single root cause.

[0044] In addition, the motion frame intervals caused by joint deviations were calculated separately for each root. ,in and These are the start and end frames where the instantaneous deviation value of the joint continuously exceeds a preset deviation threshold (which can be set to 8°).

[0045] S4, Root Cause Joint Error Correction Matching and Error Correction Output.

[0046] According to the root cause joint and the action frame interval where the deviation occurs The system retrieves the built-in sports movement professional error correction knowledge base. In this embodiment, the sports movement professional error correction knowledge base contains multiple error correction records, and each error correction record is associated with at least the following three core attribute fields: root cause joint tag. Action phase tags and the corresponding set of error correction content The error correction content set includes at least correction guidance text. Demonstration Action Diagram Labels and voice command encoding The error correction information in three modalities is used to correct the students' target physical movements.

[0047] Each target sport movement is pre-divided into multiple movement phases. For example, shot put can be divided into "preparation phase," "power generation phase," "follow-through phase," and "recovery phase." Then, the root joints are calculated. Action frame interval where deviation occurs Median frame number Then query the action phase segmentation boundary set of the target sports action. ,in This refers to the boundary frame number of each action phase. The total number of action phases, when Then determine the root cause joint at this time. The action frame interval where the deviation occurred belongs to the first... Each action phase. Determining phase affiliation by using the median frame instead of the start or end frame effectively avoids misjudgments of phases due to single-frame anomalies, thus improving the robustness of phase mapping.

[0048] Then you can determine the root cause of the joint. The system searches and matches the corresponding action stages in the sports movement error correction knowledge base to find the corresponding error correction content set, and obtains the corresponding correction guidance text, demonstration action diagram, and voice command code. Then, the correction guidance text is superimposed on the corresponding joint position of the original video screen, and the demonstration action diagram is simultaneously superimposed on the original video screen to play the corresponding demonstration action synchronously. At the same time, the voice command code is broadcast to output voice commands for guidance, thus outputting a complete error correction prompt.

[0049] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0050] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. Since the above embodiments are substantially similar to the method embodiments, their descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0051] The above embodiments provide a detailed description of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for intelligent error correction of physical education movements in primary and secondary school students based on multimodal perception, characterized in that, The method includes the following steps: S1. Collect data on multiple modalities of students performing target sports movements, calculate the angle value corresponding to each modal data, and obtain the fused angle value through weighted fusion. The data of multiple modalities includes at least video stream data and sensor data. S2. Based on the type identifier of the current target sports movement, retrieve the standard time sequence template of the corresponding movement from the pre-constructed standard sports movement dataset for primary and secondary school students, and calculate the instantaneous deviation value of each joint in any frame based on the fused angle value and the angle value of the corresponding frame in the standard time sequence template. S3. Preconstruct a directed graph model of the human motion joint chain corresponding to the target sports movement, and perform deviation backpropagation in the opposite direction of the directed graph model of the human motion joint chain according to the instantaneous deviation value, calculate the root cause confidence score of each node, and identify the root cause joint and the action frame interval in which the deviation occurs. S4. Based on the root cause joint and the action frame interval where the deviation occurs, retrieve the set of error correction content that matches the target sports movement of the student from the pre-built sports movement professional error correction knowledge base.

2. The intelligent error correction method for sports movements according to claim 1, characterized in that, In step S1, the angle value is calculated and weighted fusion is performed, including: Obtain the angle values ​​corresponding to the video stream data and sensor data respectively; The dynamic correction amount for any given frame is calculated based on the confidence level of the video stream acquisition device and the sensor signal quality. The calculation formula is as follows: In the above formula, The dynamic correction amount for frame t is determined by both the confidence level of the video stream acquisition device and the sensor signal quality for frame t. The confidence score of the visual human keypoint detection algorithm output in frame t is given. This represents the signal-to-noise ratio of the sensor signal in frame t. The preset visual confidence threshold, The preset sensor signal-to-noise ratio threshold. Set constraints for the weighting, and calculate the weighting of the video stream data and sensor data based on the dynamic correction amount and the preset base weights. The calculation formula is as follows: and The following constraints must be satisfied: In the above formula, and These are the weighted weights for video stream data and sensor data, respectively. and They are respectively and The corresponding base weights, and They are respectively and The corresponding weighting coefficients, and These are the preset minimum limits for visual weights and sensor weights, respectively; The angle values ​​corresponding to the video stream data and sensor data are weighted and fused together, and constraints are applied to obtain the fused angle value. The calculation formula is as follows: in, This represents the angle value of the j-th joint after weighted fusion in frame t. This represents the angle value of the j-th joint calculated from the video stream data in frame t. This represents the angle value of the j-th joint calculated from sensor data in frame t.

3. The intelligent error correction method for sports movements according to claim 1, characterized in that, In step S3, the directed graph model of the human motion joint chain , where the set of nodes The set of directed edges corresponding to the N joints involved in this action According to the definition of the direction of force transmission between human joints, for a set of directed edges Each directed edge in Associated with the transmission frame delay parameter and force transmission coefficient .

4. The intelligent error correction method for sports movements according to claim 3, characterized in that, The transmission frame delay parameter Based on a pre-constructed dataset of standard physical education movements for primary and secondary school students, the specific steps are as follows: When performing the target physical activity based on the centralized standard human body model in the primary and secondary school students' physical education standard movement dataset, the upstream node... and downstream nodes The frame sequence number that reaches peak angular velocity is used to calculate the propagated frame delay parameter. The benchmark value ; According to the transmission frame delay parameter The benchmark value and from upstream node to downstream nodes Limb length ratio factor The product of the propagation frame delay parameters is calculated. ; The limb length ratio factor Based on the upstream node of the student to be tested to downstream nodes The actual length of the limb segments between the corresponding joints, and the upstream node of the standard human body model. to downstream nodes The ratio of the actual lengths of the limb segments between the corresponding joints is determined.

5. The intelligent error correction method for sports movements according to claim 1, characterized in that, The force transmission coefficient Based on a pre-constructed dataset of standard physical education movements for primary and secondary school students, the specific steps are as follows: When performing the target physical activity based on the centralized standard human body model in the primary and secondary school students' physical education standard movement dataset, the upstream node... and downstream nodes Calculate the peak angular velocity value and the force transmission coefficient. The benchmark value ; Based on the upstream node when the student is performing the target physical movement, and downstream nodes The ratio of the actual peak angular velocity value and the force transmission coefficient The benchmark value Calculate the deviation coefficient of the ratio of the actual peak angular velocity of the student under test to the standard reference value. ; The body mass index correction factor is calculated based on the ratio of the body mass index of the standard human body model in the primary and secondary school students' physical education standard movement dataset to the actual body mass index of the students being tested. Based on the force transmission coefficient The benchmark value The force transmission coefficient is calculated by multiplying the deviation coefficient of the ratio of the actual peak angular velocity of the student to be tested relative to the standard reference value and the body mass index correction factor. .

6. The intelligent error correction method for sports movements according to claim 3, characterized in that, In step S3, the specific steps for calculating the root cause confidence score are as follows: Perform deviation backpropagation in the reverse direction of the directed graph model of the human motion joint chain, for each directed edge. According to the transmission frame delay parameter and force transmission coefficient Calculate upstream node For downstream nodes Expected deviation contribution value in frame t ; For each node Based on the expected deviation contribution value Calculate the root cause confidence score The calculation formula is as follows: in, Represents a node The maximum instantaneous deviation value during the entire target athletic movement. Indicates the basic weighting coefficient. Indicates the total number of frames. Represents a node The set of direct downstream nodes, Indicates the backtracking attenuation coefficient. This indicates the location weight of the downstream node.

7. The intelligent error correction method for sports movements according to claim 1, characterized in that, In step S3, the specific steps for determining the root cause joint and the motion frame interval where the deviation occurs are as follows: Obtain the root cause confidence score corresponding to each joint node, and calculate the difference ratio coefficient between each root cause confidence score and the confidence scores of the other root causes respectively. Check in turn whether each difference ratio coefficient is greater than the preset dynamic threshold parameter; If so, the joint node corresponding to the highest root cause confidence score will be the unique root cause joint. If not, then all joint nodes corresponding to the difference ratio coefficient are considered root cause joints. The starting and ending frames of each root joint where the instantaneous deviation value continuously exceeds the preset deviation threshold are obtained to construct the action frame range in which the deviation occurs.

8. The intelligent error correction method for sports movements according to claim 1, characterized in that, Step S4 specifically includes the following steps: A pre-built professional error correction knowledge base for sports movements contains multiple error correction records, with each record associated with at least one root cause joint tag. Action phase tags and the corresponding set of error correction content Core attribute fields; Each target sports movement is pre-divided into multiple movement stages, and the movement stage to which each movement frame interval due to joint deviation belongs is determined. The system searches and matches the root joints and their corresponding movement stages in the sports movement error correction knowledge base to obtain a set of error correction content for correcting students' target sports movements.

9. The intelligent error correction method for sports movements according to claim 8, characterized in that, The error correction content set shall at least include correction guidance text. Demonstration Action Diagram Labels and voice command encoding Error correction information for three modalities, and the correction guidance text. The illustrated action symbols are used to overlay and display the corresponding joint positions on the original video frame. The voice command encoding is used to overlay and display the corresponding demonstration actions on the original video screen for synchronized playback. Used to overlay onto the original video frame to broadcast the voice commands via voice.

10. A system for implementing the intelligent error correction method for sports movements according to any one of claims 1-9, characterized in that, It includes a processor and a memory, the memory being used to store a computer program, which, when executed by the processor, implements the intelligent error correction method for sports movements as described in any one of claims 1-9.

Images