A ping-pong ball intelligent learning method based on a depth camera

Abstract

The application discloses a table tennis intelligent learning method based on a depth camera and belongs to the technical field of sports education. A depth camera is used to collect a full-body action image sequence of a learner, a depth learning posture estimation algorithm is used to extract three-dimensional coordinates of human body key points and to calculate joint rotation angle data, a foot pressure sensing array is used to collect foot bottom pressure distribution data, a six-axis inertial sensing unit arranged in a racket is used to collect spatial motion parameters of the racket, the above data is input into a pre-constructed biomechanics dynamic chain model, action parameters are calculated, the action parameters are compared with a standard action parameter database in real time, action deviation scores and correction prompt information are generated, and the learner is shown the standard action evaluation results and error correction suggestions in the form of a three-dimensional animation. The application solves the problems that the prior art cannot realize accurate measurement of three-dimensional angles of full-body joints under zero-wearing conditions and lacks real-time calculation capability of dynamic chain mechanical parameters.

Description

Technical Field

[0001] This invention relates to the field of sports education technology, and in particular to a table tennis intelligent learning method based on a depth camera. Background Technology

[0002] Table tennis demands extremely high levels of precision in movement and coordination of the power chain. Different techniques correspond to different combinations of joint angles, weight transfer patterns, and ball-hitting biomechanical parameters. Taking the forehand topspin and backhand push as examples, the former requires a large rotation of the torso to generate whipping power from the upper limbs; the difference in rotation angle between the hip and shoulder joints is a key indicator of movement quality. The latter emphasizes the forward and backward movement of the forearm and fine-tuning control of the wrist, with a relatively smaller but faster weight transfer. These biomechanical differences between techniques mean that the same learner will have different error patterns in different movements, requiring precise feedback on specific movement parameters rather than general qualitative evaluations. Therefore, obtaining quantitative motion data on all elements of the learner's body during the stroke, including joints, center of gravity, and racket, and comparing it meticulously with standard movements, is crucial for improving the scientific rigor and efficiency of table tennis training.

[0003] Existing table tennis assisted training technologies mainly fall into two categories: one is based on high-speed cameras or ordinary RGB cameras, using image processing or deep learning algorithms to identify the learner's posture. This type of approach typically only outputs posture estimation results in a two-dimensional plane. Because monocular RGB cameras lack depth dimension, they cannot accurately reconstruct the rotation angles of human joints in three-dimensional space, let alone acquire key biomechanical parameters such as pelvic rotation and center of gravity position. The other type uses wearable inertial sensor devices, with multiple inertial measurement units worn on different parts of the learner's limbs to collect motion data. However, multiple sensors need to be worn on different parts of the body, restricting the learner's movement and disrupting the natural feel of the table tennis swing. More importantly, existing technologies generally lack quantitative analysis capabilities at the biomechanical level; none have established a rigorous kinetic chain mechanics model, making it impossible to calculate the torque transmission efficiency at each joint or locate weak points in the kinetic chain. Summary of the Invention

[0004] The embodiments of the present invention provide a table tennis intelligent learning method based on a depth camera, which aims to solve the problems that existing table tennis assisted training technologies cannot achieve accurate measurement of the three-dimensional angles of the whole body joints under zero-wear conditions, lack the ability to calculate mechanical parameters in real time based on kinetic chain theory, and thus make it difficult for learners to obtain accurate and interpretable quantitative feedback of movement.

[0005] To achieve the above objectives, this invention provides a table tennis intelligent learning method based on a depth camera, comprising the following steps: The learner's full-body motion image sequence is acquired by a depth camera, and the three-dimensional coordinates of the learner's human body key points are extracted based on a deep learning pose estimation algorithm. The rotation angle data of each joint is calculated based on the three-dimensional coordinates of the human body key points. The learner's plantar pressure distribution data is collected by a foot pressure sensor array, and the spatial motion parameters of the racket are collected by a six-axis inertial sensor unit set inside the racket. The rotation angle data of each joint, the plantar pressure distribution data, and the racket spatial motion parameters are input into a pre-constructed biomechanical kinetic chain model. The learner's motion parameters are calculated through the kinetic chain model. The motion parameters include pelvic rotation angle, center of gravity position change trajectory, couple state parameters, racket contact position and impulse parameters, and / or racket angle parameters. The motion parameters are compared with a preset standard motion parameter database in real time to generate motion deviation scores and correction prompts. Based on the movement deviation score, incorrect movements and standard movement demonstrations are presented to learners in the form of 3D animation, and the deviation score and correction prompts are fed back in real time through a display device.

[0006] Furthermore, the calculation of the rotation angle data of each joint based on the three-dimensional coordinates of the key human body points includes the following steps: The full-body motion image sequence captured by the depth camera is subjected to frame-by-frame human body detection. The deep learning pose estimation algorithm outputs the two-dimensional coordinates of each joint of the shoulder, elbow, wrist, hip, knee and ankle and key points of the torso on the image plane. Combined with the depth value corresponding to each pixel, the two-dimensional coordinates are back-projected to three-dimensional space through the camera intrinsic parameter matrix to obtain the three-dimensional coordinate sequence of human body key points. According to the topology of the human skeleton, the three-dimensional coordinate sequence of key points of the human body is connected by joints to construct the direction vector of each limb segment, which includes the upper arm segment, forearm segment, thigh segment, lower leg segment and trunk segment; The rotation angles of each joint are calculated based on the spatial angle between the direction vectors of adjacent limb segments. The rotation angles include the shoulder joint flexion-extension angle, shoulder joint abduction-adduction angle, elbow joint flexion-extension angle, wrist joint flexion-extension angle, hip joint flexion-extension angle, knee joint flexion-extension angle, ankle joint flexion-extension angle, trunk forward tilt angle, and trunk lateral tilt angle. A local coordinate system is established with the learner's pelvic center point as the origin, and the rotation angles of each joint are transformed and expressed in the local coordinate system.

[0007] Furthermore, the foot pressure sensing array is a thin-film pressure sensor array distributed on the training insole or training mat to collect dynamic pressure distribution data of different areas of the learner's feet, and the six-axis inertial sensing unit includes a three-axis accelerometer and a three-axis gyroscope.

[0008] Furthermore, the biomechanical kinetic chain model includes the following construction steps: Multiple sets of training data were collected from a predetermined number of professional table tennis players performing standard technical movements. Each set of training data included input data and standard values ​​of movement parameters corresponding to the input data. The input data included time-series data of joint rotation angles, time-series data of plantar pressure distribution, and time-series data of racket spatial motion parameters. The standard values ​​of movement parameters included time-series data of pelvic rotation angles, time-series data of center of gravity position change trajectory, time-series data of couple state parameters, time-series data of racket contact position and impulse parameters, and / or time-series data of racket angle parameters. The input data and standard values ​​of motion parameters in each training data set are normalized on the time axis. The dynamic time warping algorithm is used to align multiple sets of data to a uniform time axis length, and the amplitude of each parameter dimension is normalized. A multi-rigid-body skeleton model of the human body is established based on the anatomical structure of the human skeleton. The multi-rigid-body skeleton model of the human body includes multiple limb segments, and the limb segments are connected by joints with degrees of freedom. Based on the aforementioned human multi-rigid-body skeleton model, the mass distribution parameters, center of mass position parameters, and moment of inertia parameters of each limb segment are determined. Combined with the normalized joint rotation angle time series data, the position and velocity time series data of the center of mass of each limb segment in three-dimensional space are calculated through forward kinematics. The normalized plantar pressure distribution time series data is used as the bottom boundary condition of the human multi-rigid-body skeleton model. Combined with the inertial parameters and motion parameters of each limb segment, the inverse dynamics recursive algorithm is used to calculate the resultant force and resultant torque at each joint step by step in the order from distal to proximal or from proximal to distal. A complete human kinetic chain mechanical transmission mapping relationship from the sole of the foot to the end of the racket is established, and the motion parameter estimation value corresponding to the standard value of the motion parameter is calculated based on the mapping relationship. The calculated values ​​of the motion parameters are compared with the standard values ​​of the motion parameters, and the deviation between the two is calculated. When the deviation exceeds a preset threshold, the parameters of the human multi-rigid-body skeleton model are reverse-optimized and corrected until the deviation converges to within the preset threshold, thus obtaining the optimized biomechanical kinetic chain model.

[0009] Furthermore, the action deviation score includes the following generation steps: Align all acquired motion parameters with the corresponding standard motion parameters in the preset standard motion parameter database on the timeline. Calculate the deviation between each motion parameter and the corresponding standard motion parameter; The deviation value is input into a pre-trained motion scoring neural network model, which uses multiple sets of known deviation values ​​and their corresponding expert scores as training samples to output a comprehensive motion deviation score.

[0010] Furthermore, the correction prompt information includes motion deviation positioning information, deviation direction indication information, correction guidance information, and standard motion comparison information; wherein, the motion deviation positioning information is used to identify the name of the motion parameter that exceeds the standard and the corresponding body part; the deviation direction indication information is used to indicate the deviation direction of the current motion parameter relative to the standard motion parameter; the correction guidance information is used to indicate the adjustment direction and adjustment range required to adjust the current motion parameter to the standard value; and the standard motion comparison information is used to display the comparison between the current motion posture and the standard motion posture in the form of 3D animation or text description.

[0011] Furthermore, the preset standard motion parameter database contains standard motion templates categorized by different difficulty levels and different technical motion types. Each standard motion template includes time-series curves of joint rotation angles, center of gravity movement trajectory, pelvic rotation trajectory, racket angle time-series curves, and reference values ​​for ball impact.

[0012] Furthermore, the demonstration of incorrect and standard movements to learners in the form of 3D animation includes: Acquire learners' real-time motion parameters and map them to generate 3D skeletal model animations; On the same display interface, the 3D skeletal model animation of the learner's current action and the 3D skeletal model animation of the standard action are displayed side by side in a synchronous manner. The deviation vector between the learner's current joint spatial position and the corresponding joint spatial position of the standard movement is calculated. In the 3D skeletal model animation, a deviation indicator line is generated from the learner's current joint position to the standard joint position. Each joint and limb segment is marked with a color gradient according to the magnitude of the deviation.

[0013] The above technical solution has the following technical effects: (1) This invention uses a depth camera combined with a deep learning pose estimation algorithm to accurately extract the three-dimensional rotation angles of all joints in the body without the learner needing to wear any markers or sensors. This eliminates the constraints of traditional wearable devices on the learner's natural movements and ensures the real feel of table tennis training.

[0014] (2) By constructing a biomechanical kinetic chain model based on anatomy and inverse dynamics, this invention integrates joint rotation angle, plantar pressure distribution and racket spatial motion parameters, realizing a complete mechanical transmission mapping from the sole of the foot to the end of the racket. It can quantitatively calculate the pelvic rotation angle, the trajectory of the center of gravity position change, the couple state parameters, the racket contact position and impulse parameters and the racket angle parameters, thus making up for the shortcomings of existing technologies that cannot locate the weak links of the kinetic chain.

[0015] (3) This invention compares the motion parameters with the standard motion parameter database in real time and uses a pre-trained neural network model to generate a comprehensive motion deviation score. This overcomes the shortcomings of traditional rule-weighted scoring, which relies on manual weight setting and cannot capture the nonlinear coupling relationship between parameters, making the scoring results closer to expert judgment.

[0016] (4) This invention transforms abstract motion deviation data into intuitive visual information through synchronous comparison display of three-dimensional animation and deviation indicator lines. Learners can understand the adjustment direction and range of each joint and limb segment without the need for a coach to be present, thus reducing the threshold for self-learning.

[0017] (5) This invention solidifies the standard movement parameters of professional table tennis players into a database in the form of data, transforming the "correct movements" that were originally confined to the individual experience of coaches and were difficult to quantify and transmit into quantifiable, calculable, and comparable digital assets. Based on this, this invention uses 3D animation as the core medium of expression, mapping abstract movement deviation data in real time into a skeletal model comparison screen and deviation indicator lines that learners can intuitively perceive, breaking through the information transmission bottleneck of traditional text descriptions or static illustrations in movement teaching. More importantly, this invention divides all training courses into several levels arranged from low to high difficulty, each level corresponding to a standard technical movement and having a clear passing score threshold. Learners must obtain a passing score before unlocking the next level. Learners do not need to rely on the real-time on-site guidance of coaches; they only need to follow the 3D animation demonstration, adjust their own movements according to the deviation indicator lines, and judge their own progress based on the score feedback to independently complete the complete training loop from beginner to advanced levels under data-driven and visual guidance. This product format frees the teaching of fine motor skills, which has long been considered to rely on the verbal and physical instruction of coaches, from dependence on the individual experience of coaches, opening up a new and feasible path for the self-learning and popularization of human motor skills. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating a method for nutritional intervention in chronic diseases based on multidimensional lifestyle data analysis, according to an embodiment of the present invention. Detailed Implementation

[0019] To further illustrate the various embodiments, the present invention provides accompanying drawings. These drawings are part of the disclosure of the present invention, primarily used to illustrate the embodiments and to explain the operating principles of the embodiments in conjunction with the relevant descriptions in the specification. With reference to these drawings, those skilled in the art should be able to understand other possible implementations and the advantages of the present invention. Components in the drawings are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0020] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments.

[0021] Figure 1 This is a flowchart illustrating a method for nutritional intervention in chronic diseases based on multidimensional lifestyle data analysis, according to an embodiment of the present invention. Figure 1 As shown, the method of this embodiment includes the following steps: The learner's full-body motion image sequence is acquired by a depth camera, and the three-dimensional coordinates of the learner's human body key points are extracted based on a deep learning pose estimation algorithm. The rotation angle data of each joint is calculated based on the three-dimensional coordinates of the human body key points.

[0022] The depth camera, employing an Intel RealSense D435 or equivalent performance, is positioned 3 to 5 meters directly in front of the learner, with the lens optical axis approximately perpendicular to the learner's sagittal plane. The camera's height above the ground is approximately 1.2 to 1.5 meters to cover the learner's full range of motion. The depth camera includes an RGB camera and an infrared depth sensor, simultaneously outputting color and depth images at a frame rate of at least 30fps and a depth image resolution of at least 640×480 pixels. After deployment, the Zhang Zhengyou calibration method is used to calibrate the depth camera's intrinsic parameters, obtaining the camera's intrinsic parameter matrix and distortion coefficients. Simultaneously, a multi-view calibration board is used to calibrate the camera's extrinsic parameters in the world coordinate system, including rotation matrices and translation vectors. Once calibration is complete, the transformation relationship from the camera coordinate system to the world coordinate system is fixed, and all subsequent calculations are performed in the world coordinate system.

[0023] The sequence of full-body motion images captured by the depth camera is processed frame by frame. A deep learning-based pose estimation algorithm is used to perform human detection and keypoint localization on the RGB images. The pose estimation algorithm employs HRNet, which, through a parallel multi-resolution subnet structure, fuses multi-scale information while maintaining high-resolution feature maps, outputting the two-dimensional coordinates of the shoulder, elbow, wrist, hip, knee, and ankle joints, as well as key points of the torso, on the image plane. Simultaneously, the depth values ​​corresponding to each pixel are read from the depth image. For any keypoint, its two-dimensional coordinates on the image plane, the corresponding depth value, and the camera intrinsic parameter matrix are used to backproject it into three-dimensional space. After traversing all keypoints, a three-dimensional coordinate sequence of human keypoints is obtained. Depth information is introduced for three-dimensional backprojection because two-dimensional image coordinates only provide orientation information and lack the distance dimension; the depth values ​​provided by the depth camera can complete this dimension, thus elevating the planar coordinates to three-dimensional spatial coordinates.

[0024] After obtaining the 3D coordinate sequence of key human body points, the joints are connected according to the topology of the human skeleton to construct the direction vectors of each limb segment. Limb segments include the upper arm, forearm, thigh, lower leg, and trunk. Taking the upper arm segment as an example, its direction vector points from the 3D coordinates of the shoulder joint to the 3D coordinates of the elbow joint; the direction vector of the forearm segment points from the 3D coordinates of the elbow joint to the 3D coordinates of the wrist joint. The rotation angles of each joint are calculated based on the spatial angles between the direction vectors of adjacent limb segments. Rotation angles include the shoulder flexion-extension angle, shoulder abduction-adduction angle, elbow flexion-extension angle, wrist flexion-extension angle, hip flexion-extension angle, knee flexion-extension angle, ankle flexion-extension angle, trunk forward tilt angle, and trunk lateral tilt angle. A local coordinate system is established with the learner's pelvic center point as the origin. The pelvic center point is defined as the midpoint between the left and right hip key points. A right-handed coordinate system is formed by using the direction from the center of the pelvis to the key points of the trunk as the positive Y-axis, the direction from the center of the pelvis to the key points of the left or right hip as the positive X-axis, and the cross product of the X and Y axes as the positive Z-axis. The rotation angles of each joint are expressed in the local coordinate system by transforming them from the world coordinate system to the local coordinate system. This transformation includes translation and rotation. The reason for establishing the local coordinate system is that learners may move in space during training. If joint angles are expressed in the world coordinate system, changes in the learner's overall position will affect the calculation results. The local coordinate system, based on the learner's own pelvis, can eliminate the interference of overall displacement and orientation changes on the joint angle calculation.

[0025] The learner's plantar pressure distribution data is collected by a foot pressure sensor array, and the spatial motion parameters of the racket are collected by a six-axis inertial sensor unit set inside the racket.

[0026] In one specific implementation, the foot pressure sensing array is a pressure sensor array distributed within the sole of the shoe, collecting dynamic pressure distribution data of various areas of the learner's feet. When the foot pressure sensing array is placed inside the sole, thin-film pressure sensors are embedded in the forefoot, arch, and heel areas of each sole, with each sole containing no fewer than 8 sensing points. The sampling frequency of the foot pressure sensing array is no less than 100Hz. The instantaneous pressure center coordinates of the learner's feet are calculated based on the pressure value and spatial position of each sensing point. The instantaneous pressure center coordinates describe the position of the point of application of the sole pressure on the plane of the foot pressure sensing array, and its physical meaning is the projection position of the learner's body center of gravity on the support surface of the foot. By connecting the instantaneous pressure center coordinates of consecutive moments in time sequence, the trajectory of the instantaneous pressure center change within a preset time window is obtained. The six-axis inertial sensing unit includes a three-axis accelerometer and a three-axis gyroscope, which are fixedly installed inside the racket handle or in a groove on the back of the racket, and rigidly connected to the racket by adhesive or threaded fixing. The sampling frequency of the six-axis inertial sensing unit is no less than 200Hz. A three-axis accelerometer measures the linear acceleration of the racket in three-dimensional space, and a three-axis gyroscope measures the angular velocity of the racket around the three coordinate axes. Time integration of the acceleration and angular velocity data yields the racket's velocity, displacement, and attitude angle in three-dimensional space. The racket's spatial motion parameters are crucial inputs for subsequent end-point verification of the kinetic chain model. By comparing the theoretical racket motion parameters calculated by the model with the actual racket motion parameters measured by the six-axis inertial sensing unit, closed-loop correction of the model can be performed.

[0027] The rotation angle data of each joint, the plantar pressure distribution data, and the racket spatial motion parameters are input into the pre-constructed biomechanical kinetic chain model. The learner's motion parameters are calculated through the kinetic chain model. The motion parameters include the force of the fingers gripping the racket, the pelvic rotation angle, the trajectory of the center of gravity position change, the couple state parameters, the racket contact position and impulse parameters, and the racket angle parameters.

[0028] Before being used online, biomechanical kinetic chain models need to be constructed and optimized in advance. The construction process is as follows: The process involves collecting multiple sets, such as 1000 sets, of training data from professional table tennis players performing standard technical movements. In one specific implementation, the standard technical movements include at least forehand attack, backhand attack, forehand topspin, backhand topspin, forehand chop, backhand chop, forehand serve, backhand serve, forehand smash, and backhand push. The data collection process for each set of training data is consistent with the aforementioned steps, namely, simultaneously acquiring depth camera images, foot pressure sensor data, and six-axis inertial sensor data. Each set of training data includes input data and corresponding standard values ​​of motion parameters. The input data includes time-series data of joint rotation angles, time-series data of plantar pressure distribution, and time-series data of racket spatial motion parameters. The standard values ​​of motion parameters are obtained as follows: an expert group consisting of at least three table tennis coaches with national-level or higher qualifications independently evaluates the same set of raw sensor data collected, and marks the pelvic rotation angle, center of gravity position, torque state, racket contact position and impulse, and racket angle at each moment. The arithmetic mean of the three marking results is taken as the standard value of motion parameters at that moment.

[0029] The input data and standard values ​​of motion parameters in each training dataset were normalized over time. Since different athletes perform the same technical movement at different speeds, the time span of the movement varies, and direct cross-sample comparisons can lead to timeline misalignment. A dynamic time warping algorithm was used to align multiple datasets to a unified timeline length. This algorithm constructs an optimal path through dynamic programming, minimizing the cumulative distance between two time series along this path, thus aligning motion sequences with different speeds over time and enabling subsequent statistical modeling to be performed on a unified time reference. Amplitude normalization was performed on each parameter dimension, mapping the value range of each parameter to a unified interval and eliminating the influence of different parameters' dimensions on the modeling process.

[0030] A multi-rigid-body skeletal model of the human body is established based on the anatomical structure of the human skeleton. This model comprises multiple limb segments connected by joints with degrees of freedom. Joints include at least the shoulder, elbow, wrist, hip, knee, and ankle joints. Each limb segment is treated as a rigid body, and its elastic deformation during movement is ignored. For each limb segment, its mass distribution parameters, center of mass position parameters, and moment of inertia parameters are determined based on anthropometry statistics. Anthropometry statistics can be obtained by referring to the human inertia parameter tables provided in "Anthropometric Dimensions of Chinese Adults" (GB / T 10000-2023) or the Winter biomechanics textbook. The proportion of each limb segment's mass to the total weight, the proportion of the center of mass located proximally to the limb segment, and the moment of inertia around the center of mass are obtained by referring to tables based on the learner's height and weight.

[0031] By combining normalized joint rotation angle time-series data, forward kinematics is used to calculate the position and velocity time-series data of the center of mass of each limb segment in three-dimensional space. The basic idea of ​​forward kinematics is: given the initial pose of the human multi-rigid-body skeleton model and the time-series data of each joint rotation angle, starting from the root node, the position and orientation of each limb segment in space are calculated step-by-step according to the skeletal topology. Taking the pelvic segment as the root node, the spatial pose of the pelvic segment is first determined based on the position of the pelvic center point. Then, the spatial pose of the thigh segment is calculated based on the hip joint rotation angle, and then the spatial pose of the lower leg segment is calculated based on the knee joint rotation angle, and so on. Once the spatial pose of each limb segment is determined, the position of the center of mass of each limb segment is also determined. Time difference analysis is performed on the center of mass position at each moment to obtain the time-series data of the center of mass velocity.

[0032] Normalized plantar pressure distribution time-series data was used as the bottom boundary conditions for the human multi-rigid-body skeleton model. Plantar pressure distribution reflects the contact mechanics between the learner and the ground, including the magnitude and location of the ground reaction force in each area of ​​the foot. Combining the inertial and kinematic parameters of each limb segment, an inverse dynamics recursive algorithm was employed to calculate the resultant force and torque at each joint in a step-by-step manner from distal to proximal. The basic principle of inverse dynamics recursion is: starting from the distal limb segment, based on the kinematic parameters and external forces acting on that segment, the required resultant force and torque at its proximal joint are calculated using the Newton-Euler equations, and then this process is repeated step-by-step upwards along the kinetic chain. Taking a forehand attack in table tennis as an example, the resultant force and torque at the wrist joint are first calculated based on the racket's acceleration and the forces acting on it. Then, the resultant force and torque at the elbow joint are calculated using the kinematic parameters of the forearm segment, followed by the shoulder joint using the kinematic parameters of the upper arm segment, and so on until the hip and knee joints. By iteratively building up step by step, a complete kinetic chain mapping relationship of the human body is established from the sole of the foot to the end of the racket. Based on this mapping relationship, by inputting the joint rotation angle, sole pressure distribution, and racket spatial motion parameters at any given moment, the calculated motion parameters at that moment can be obtained, including pelvic rotation angle, center of gravity position, couple state, racket contact position and impulse, and racket angle.

[0033] The calculated motion parameters are compared with the standard values ​​to calculate the deviation. Deviation calculation is performed for each parameter dimension separately, using root mean square error (RMSE) as the deviation metric. When the deviation exceeds a preset threshold, the parameters of the human multi-rigid-body skeleton model are reverse-optimized and corrected. The reverse-optimization corrects the mass distribution parameters, center-of-mass position parameters, and moment of inertia parameters of each limb segment. The correction process uses a gradient descent optimization algorithm, with the sum of squared deviations as the loss function. The gradient of the loss function with respect to each parameter to be corrected is calculated using the chain rule, and the parameter values ​​are updated along the gradient descent direction. The updated parameters are then recalculated using forward kinematics and inverse dynamics to obtain new calculated motion parameters, which are again compared with the standard values ​​to calculate the deviation. This iterative process is repeated until the deviation converges to within the preset threshold, resulting in the optimized biomechanical kinetic chain model.

[0034] The motion parameters are compared with a preset database of standard motion parameters in real time to generate motion deviation scores and correction prompts.

[0035] The standard movement parameter database contains standard movement templates categorized by difficulty level and technical movement type. Difficulty levels are divided into three levels: beginner, intermediate, and advanced. Beginner corresponds to mastering the basic movement framework, intermediate corresponds to improving movement precision, and advanced corresponds to movement stability under high-speed and high-intensity conditions. Each standard movement template includes timing curves for joint rotation angles, center of gravity movement trajectory, pelvic rotation trajectory, racket angle timing curves, and reference values ​​for impact impulse. The standard movement templates are derived from statistical results of training data from professional table tennis players, specifically by averaging the standard values ​​of multiple athletes of the same age and gender for the same technical movement.

[0036] The comparison process is implemented as follows: All acquired motion parameters are time-aligned with the corresponding standard motion parameters in the standard motion parameter database. Time alignment employs a dynamic time warping algorithm to eliminate the time difference between the learner's motion speed and the standard template's motion speed. After time alignment, the deviation value between each motion parameter and its corresponding standard motion parameter is calculated. Taking joint rotation angle as an example, the absolute value of the difference between the measured angle and the standard angle at each moment is calculated, and then the root mean square is taken along both the joint and time dimensions to obtain the deviation value for that joint. Other parameters, such as center of gravity position, pelvic rotation angle, and racket angle, have their respective deviation values ​​calculated in the same way. These deviation values ​​are then input into a pre-trained motion scoring neural network model. The motion scoring neural network model uses a three-layer fully connected network structure. The number of nodes in the input layer equals the number of deviation values, the hidden layer contains 64 nodes with the ReLU activation function, and the output layer has one node, outputting a comprehensive motion deviation score. This network is pre-trained using multiple sets of known deviation values ​​and their corresponding expert scores as training samples. The expert scores were obtained by averaging the scores given independently to the learners' movements by a panel of three national-level coaches, with a score range of 0 to 100. The goal of network training was to minimize the difference between the network's output comprehensive deviation score and the expert scores. After training, the network could automatically output a comprehensive movement deviation score based on the input deviation values ​​for each dimension. This score comprehensively reflected the overall degree of deviation of the learner's movements from the standard movements; a higher score indicated that the movements were closer to the standard.

[0037] The correction prompts include movement deviation location information, deviation direction indication information, correction guidance information, and standard movement comparison information. Movement deviation location information identifies the name of the movement parameter that exceeds the standard and its corresponding body part. Excessive deviation is defined as a deviation value exceeding 1.5 times the standard deviation of that parameter. Deviation direction indication information indicates the direction of deviation of the current movement parameter relative to the standard movement parameter, such as whether the shoulder flexion / extension angle is too large or too small. Correction guidance information indicates the direction and magnitude of adjustment required to bring the current movement parameter to the standard value, such as "lower the right shoulder by 2 cm" or "increase the hip tilt by 5 degrees." Standard movement comparison information uses a 3D animation to compare the current movement posture with the standard movement posture.

[0038] Based on the movement deviation score, incorrect movements and standard movement demonstrations are presented to learners in the form of 3D animation, and the deviation score and correction prompts are displayed in real time through the display device.

[0039] The system acquires the learner's real-time motion parameters and maps them to generate a 3D skeletal model animation. The 3D skeletal model is constructed based on the aforementioned human multi-rigid-body skeleton model and has the same limb segment topology as the learner. The learner's joint rotation angles are mapped in real-time to the corresponding joints in the 3D skeletal model, driving the 3D skeletal model to perform movements synchronized with the learner. On the same display interface, the 3D skeletal model animations of the learner's current movement and the standard movement are displayed side-by-side. The standard movement's 3D skeletal model animation is driven by the timing curves of standard joint angles from the standard movement parameter database. This side-by-side display allows the learner to intuitively compare the morphological differences between their own movement and the standard movement. The system calculates the deviation vector between the learner's current joint spatial position and the corresponding joint spatial position in the standard movement, generating a deviation indicator line in the 3D skeletal model animation pointing from the learner's current joint position to the standard joint position. Each joint and limb segment is labeled with a color gradient based on the magnitude of the deviation. Small deviations are marked in green, gradually transitioning to yellow and red as the deviation increases. Learners can intuitively determine which joints need adjustment, in which direction, and with what urgency, by observing the direction, length, and color changes of the deviation indicator lines.

[0040] The standard movement comparison information in the prompt message has been revised to display a 3D animation comparing the current movement posture with the standard movement posture. Specifically, the left half of the display interface shows a 3D skeletal model animation of the learner's current movement, while the right half shows a 3D skeletal model animation of the standard movement. Both skeletal models use the same perspective, size, and movement progress. By comparing the left and right sides, learners can clearly understand the differences between their own movement and the standard movement in terms of joint angles and limb positions.

[0041] The display device is positioned to the side of the depth camera or directly in front of the learner to show the deviation score and correction prompts in real time. In addition, the system provides accompanying voice training guidance via an audio output device. The content of the voice guidance is automatically generated based on the one or more movement parameters with the most significant deviation at the current moment. When the overall deviation score is higher than the passing threshold, the system announces the score result and the unlock information for the next level in an encouraging tone. When the overall deviation score is lower than the threshold, the system extracts the movement parameter with the largest deviation and its correction guidance from the correction prompts, converting it into short, conversational sentences for broadcast. For example, when the torso forward tilt angle deviation is the largest, the voice prompt is "Pay attention to the torso forward tilt angle, please straighten your back slightly"; when the shoulder abduction angle deviation is the largest, the voice prompt is "The right shoulder is not open enough, please extend it outward by about a fist's distance"; when the center of gravity is too far back, the voice prompt is "Shift the center of gravity forward, shift the pressure to the balls of your feet." The voice prompts are automatically triggered after each shot, and the content is dynamically updated according to the real-time changes in the learner's movement quality. When the same deviation appears first in the deviation ranking three times consecutively, the system repeatedly emphasizes that area to reinforce the learner's attention. The voice broadcast uses a standard Mandarin female voice with a speech rate set at 200 words per minute. Learners can also adjust the on / off state, volume, and speech rate of the voice broadcast according to their personal preferences. The purpose of voice-assisted training is to simultaneously transmit key correction information through the auditory channel while the learner visually corrects their movements by focusing on the 3D animation and deviation indicator lines on the display device, forming a closed feedback loop of visual and auditory channels. This allows learners to receive action correction instructions in real time without frequently interrupting their actions to check the screen, maintaining the continuity and immersion of training.

[0042] After a learner completes a technical movement, the system records the overall deviation score and deviation values ​​for each dimension of the movement, storing them in a local or cloud database. Learners can access historical training records at any time to view the score change curves for each training session and understand their progress in improving their movements.

[0043] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A table tennis intelligent learning method based on a depth camera, characterized in that, Includes the following steps: The learner's full-body motion image sequence is acquired by a depth camera, and the three-dimensional coordinates of the learner's human body key points are extracted based on a deep learning pose estimation algorithm. The rotation angle data of each joint is calculated based on the three-dimensional coordinates of the human body key points. The learner's plantar pressure distribution data is collected by a foot pressure sensor array, and the spatial motion parameters of the racket are collected by a six-axis inertial sensor unit set inside the racket. The rotation angle data of each joint, the plantar pressure distribution data, and the racket spatial motion parameters are input into a pre-constructed biomechanical kinetic chain model. The learner's motion parameters are calculated through the kinetic chain model. The motion parameters include pelvic rotation angle, center of gravity position change trajectory, couple state parameters, racket contact position and impulse parameters, and / or racket angle parameters. The motion parameters are compared with a preset standard motion parameter database in real time to generate motion deviation scores and correction prompts. Based on the movement deviation score, incorrect movements and standard movement demonstrations are presented to learners in the form of 3D animation, and the deviation score and correction prompts are fed back in real time through a display device.

2. The table tennis intelligent learning method based on a depth camera according to claim 1, characterized in that, The calculation of the rotation angle data of each joint based on the three-dimensional coordinates of the key points of the human body includes the following steps: The full-body motion image sequence captured by the depth camera is subjected to frame-by-frame human body detection. The deep learning pose estimation algorithm outputs the two-dimensional coordinates of each joint of the shoulder, elbow, wrist, hip, knee and ankle and key points of the torso on the image plane. Combined with the depth value corresponding to each pixel, the two-dimensional coordinates are back-projected to three-dimensional space through the camera intrinsic parameter matrix to obtain the three-dimensional coordinate sequence of human body key points. According to the topology of the human skeleton, the three-dimensional coordinate sequence of key points of the human body is connected by joints to construct the direction vector of each limb segment, which includes the upper arm segment, forearm segment, thigh segment, lower leg segment and trunk segment; The rotation angles of each joint are calculated based on the spatial angle between the direction vectors of adjacent limb segments. The rotation angles include the shoulder joint flexion-extension angle, shoulder joint abduction-adduction angle, elbow joint flexion-extension angle, wrist joint flexion-extension angle, hip joint flexion-extension angle, knee joint flexion-extension angle, ankle joint flexion-extension angle, trunk forward tilt angle, and trunk lateral tilt angle. A local coordinate system is established with the learner's pelvic center point as the origin, and the rotation angles of each joint are transformed and expressed in the local coordinate system.

3. The table tennis intelligent learning method based on a depth camera according to claim 1, characterized in that, The foot pressure sensing array is a thin-film pressure sensor array distributed on the training insole or training mat to collect dynamic pressure distribution data of different areas of the learner's feet. The six-axis inertial sensing unit includes a three-axis accelerometer and a three-axis gyroscope.

4. The table tennis intelligent learning method based on a depth camera according to claim 1, characterized in that, The biomechanical kinetic chain model includes the following construction steps: Multiple sets of training data were collected from a predetermined number of professional table tennis players performing standard technical movements. Each set of training data included input data and standard values ​​of movement parameters corresponding to the input data. The input data included time-series data of joint rotation angles, time-series data of plantar pressure distribution, and time-series data of racket spatial motion parameters. The standard values ​​of movement parameters included time-series data of pelvic rotation angles, time-series data of center of gravity position change trajectory, time-series data of couple state parameters, time-series data of racket contact position and impulse parameters, and / or time-series data of racket angle parameters. The input data and standard values ​​of motion parameters in each training data set are normalized on the time axis. The dynamic time warping algorithm is used to align multiple sets of data to a uniform time axis length, and the amplitude of each parameter dimension is normalized. A multi-rigid-body skeleton model of the human body is established based on the anatomical structure of the human skeleton. The multi-rigid-body skeleton model of the human body includes multiple limb segments, and the limb segments are connected by joints with degrees of freedom. Based on the aforementioned human multi-rigid-body skeleton model, the mass distribution parameters, center of mass position parameters, and moment of inertia parameters of each limb segment are determined. Combined with the normalized joint rotation angle time series data, the position and velocity time series data of the center of mass of each limb segment in three-dimensional space are calculated through forward kinematics. The normalized plantar pressure distribution time series data is used as the bottom boundary condition of the human multi-rigid-body skeleton model. Combined with the inertial parameters and motion parameters of each limb segment, the inverse dynamics recursive algorithm is used to calculate the resultant force and resultant torque at each joint step by step in the order from distal to proximal or from proximal to distal. A complete human kinetic chain mechanical transmission mapping relationship from the sole of the foot to the end of the racket is established, and the motion parameter estimation value corresponding to the standard value of the motion parameter is calculated based on the mapping relationship. The calculated values ​​of the motion parameters are compared with the standard values ​​of the motion parameters, and the deviation between the two is calculated. When the deviation exceeds a preset threshold, the parameters of the human multi-rigid-body skeleton model are reverse-optimized and corrected until the deviation converges to within the preset threshold, thus obtaining the optimized biomechanical kinetic chain model.

5. The table tennis intelligent learning method based on a depth camera according to claim 1, characterized in that, The motion deviation score includes the following generation steps: Align all acquired motion parameters with the corresponding standard motion parameters in the preset standard motion parameter database on the timeline. Calculate the deviation between each motion parameter and the corresponding standard motion parameter; The deviation value is input into a pre-trained motion scoring neural network model, which uses multiple sets of known deviation values ​​and their corresponding expert scores as training samples to output a comprehensive motion deviation score.

6. The table tennis intelligent learning method based on a depth camera according to claim 1, characterized in that, The correction prompt information includes motion deviation location information, deviation direction indication information, correction guidance information, and standard motion comparison information; wherein, the motion deviation location information is used to identify the name of the motion parameter that exceeds the standard and the corresponding body part; the deviation direction indication information is used to indicate the deviation direction of the current motion parameter relative to the standard motion parameter; the correction guidance information is used to indicate the adjustment direction and adjustment range required to adjust the current motion parameter to the standard value; the standard motion comparison information is used to display the comparison between the current motion posture and the standard motion posture in the form of 3D animation or text description.

7. The table tennis intelligent learning method based on a depth camera according to claim 1, characterized in that, The preset standard motion parameter database contains standard motion templates categorized by different difficulty levels and different technical motion types. Each standard motion template includes time-series curves of joint rotation angles, center of gravity movement trajectory, pelvic rotation trajectory, racket angle time-series curves, and reference values ​​for ball impact.

8. The table tennis intelligent learning method based on a depth camera according to claim 1, characterized in that, The method of demonstrating incorrect and standard movements to learners in the form of 3D animation includes: Acquire learners' real-time motion parameters and map them to generate 3D skeletal model animations; On the same display interface, the 3D skeletal model animation of the learner's current action and the 3D skeletal model animation of the standard action are displayed side by side in a synchronous manner. The deviation vector between the learner's current joint spatial position and the corresponding joint spatial position of the standard movement is calculated. In the 3D skeletal model animation, a deviation indicator line is generated from the learner's current joint position to the standard joint position. Each joint and limb segment is marked with a color gradient according to the magnitude of the deviation.

Images