Coordinated control method and system of fitness videos and training equipment

Abstract

The present application relates to the fields of sensor technology and Internet of Things, and provides a fitness video and training equipment cooperative control method and system, which comprises: collecting a cloud-based fitness teaching video stream to extract a sequence of key points of a skeleton of a demonstration action, an action intensity level and a standard motion trajectory, collecting real-time physiological data and real-time motion posture of a user; identifying a target action of each stage in the fitness teaching video stream, generating a target control parameter of a training equipment corresponding to the target action to construct a video-parameter mapping library of the training equipment; determining a physical fitness adaptation coefficient of the user, calculating an action completion degree and a motion risk coefficient of the user, and generating a safety control threshold of the training equipment; and generating a cooperative control instruction of the training equipment to execute cooperative control of the training equipment. The present application can improve the adaptability and safety of the training equipment cooperative control.

Description

Technical Field

[0001] This invention relates to a collaborative control method and system for fitness videos and training equipment, belonging to the fields of sensor technology and the Internet of Things. Background Technology

[0002] Collaborative control of fitness videos and training equipment refers to the use of intelligent technology to enable the parameters of fitness videos and training equipment to work together automatically and in real time. This provides users with a personalized, efficient, and safe training experience.

[0003] Traditional training equipment control methods use preset fixed parameter control. By setting static parameters such as the operating intensity, frequency, and duration of the training equipment, and coordinating with the fixed rhythm of pre-recorded fitness videos to complete the training, this method lacks an intelligent adaptation mechanism for the video rhythm and equipment operation because the parameters are fixed and there is no mapping relationship with the fitness video. This results in a mismatch between equipment parameters and exercise movements, leading to poor training results and safety risks. Summary of the Invention

[0004] This invention provides a method and system for the coordinated control of fitness videos and training equipment, the main purpose of which is to improve the adaptability and safety of the coordinated control of training equipment.

[0005] To achieve the above objectives, the present invention provides a method for the coordinated control of fitness videos and training equipment, comprising:

[0006] The system collects fitness instructional video streams from the cloud to extract the skeletal key point sequence, movement intensity level, and standard movement trajectory of the demonstrated movements. It also collects the user's real-time physiological data and real-time movement posture through the sensor group integrated into the training equipment. Based on the skeletal key point sequence, the target action of each stage in the fitness teaching video stream is identified. Based on the action intensity level and the standard motion trajectory, the target control parameters of the training device corresponding to the target action are generated to construct the video-parameter mapping library of the training device. Based on the real-time physiological information and the intensity level of the movement, the user's physical fitness coefficient is determined. Based on the real-time movement posture and the standard movement trajectory, the user's movement completion degree and movement risk coefficient are calculated. Combining the physical fitness coefficient, the movement completion degree and the movement risk coefficient, the safety control threshold of the training device is generated. Based on the video-parameter mapping library and the security control threshold, collaborative control instructions for the training device are generated to execute the collaborative control of the training device.

[0007] Optionally, calculating the user's action completion rate and motion risk coefficient based on the real-time motion posture and the standard motion trajectory includes: Extract the user's real-time key point sequence from the real-time motion posture, and extract the standard key point sequence of the standard motion trajectory; based on the real-time key point sequence and the standard key point sequence, extract the real-time motion features of the real-time motion posture and the standard motion features of the standard motion trajectory, respectively. Based on the real-time motion characteristics and the standard motion characteristics, the user's action completion rate and motion risk coefficient are calculated.

[0008] Optionally, the step of extracting real-time motion features of the real-time motion posture and standard motion features of the standard motion trajectory based on the real-time keypoint sequence and the standard keypoint sequence, respectively, includes: Time-normalize the real-time keypoint sequence and the standard keypoint sequence to obtain a normalized real-time keypoint sequence and a normalized standard keypoint sequence. Calculate the Euclidean distance between the regularized real-time keypoint sequence and the regularized standard keypoint sequence to determine the posture difference between the real-time motion posture and the standard motion trajectory; Calculate the real-time dynamic parameters of the regularized real-time keypoint sequence and the standard dynamic parameters of the regularized standard keypoint sequence, respectively. By combining the posture differences, the real-time dynamic parameters, and the standard dynamic parameters, the real-time motion characteristics of the real-time motion posture and the standard motion characteristics of the standard motion trajectory are determined.

[0009] Optionally, the extraction of the skeletal key point sequence, movement intensity level, and standard motion trajectory of the demonstration movement includes: Identify the demonstration subject in each frame of the fitness instruction video stream, determine the skeletal key points of the demonstration subject, and integrate the skeletal key point sequence of the demonstration movement; Based on the skeletal key point sequence, the joint movement speed, movement amplitude, and movement acceleration of the demonstrated movement are calculated, and the repetition period of the demonstrated movement is identified in order to calculate the movement frequency of the demonstrated movement. Based on the joint movement speed, the movement amplitude, the movement acceleration, and the movement frequency, the movement intensity level of the demonstration movement is determined, and the pixel coordinates of key points in the skeletal key point sequence are converted into spatial coordinates to construct the standard movement trajectory of the demonstration movement.

[0010] Optionally, the acquisition of the user's real-time physiological data and real-time motion posture through the sensor group integrated into the training device includes: Configure the multimodal sensors of the training device, construct the layout diagram and communication protocol of the multimodal sensors, and integrate the collaborative sensing network of the training device; Based on the collaborative sensing network, the user's raw physiological data and raw posture data are collected synchronously, and the raw physiological data and raw posture data are processed to obtain real-time physiological data and real-time motion posture.

[0011] Optionally, identifying the target movement for each stage of the fitness instruction video stream based on the skeletal keypoint sequence includes: Based on the skeletal key point sequence, the motion features of the fitness instruction video stream are extracted to determine the motion boundaries of the demonstrated movements in the fitness instruction video stream. Based on the action boundaries, the fitness instruction video stream is divided into fitness action stages; Identify the target movements in each stage of the fitness exercise from the fitness instruction video stream.

[0012] Optionally, generating the target control parameters of the training device corresponding to the target action based on the action intensity level and the standard motion trajectory includes: The action intensity level is converted into a target physical quantity, and the basic load for completing the target action is determined. The kinematic parameters of the standard motion trajectory are extracted, and the standard motion trajectory is discretized to obtain a time-space coordinate sequence. Calculate the time-space resistance sequence of the time-space coordinate sequence based on the kinematic parameters, the target physical quantity, and the basic load; Based on the time-space resistance sequence, the target velocity curve, soft limit parameters, and hard limit parameters of the training device are constructed to generate the target control parameters of the training device.

[0013] Optionally, determining the user's fitness coefficient based on the real-time physiological information and the intensity level of the movement includes: Based on the real-time physiological information, the user's real-time physiological characteristic values ​​are determined; Based on the user's personal profile, the target physiological characteristic value and physiological safety limit of the user in the action intensity level are calculated. Based on the real-time physiological characteristic value, the target physiological characteristic value and the physiological safety limit, the user's physical fitness adaptation coefficient is calculated.

[0014] Optionally, generating the safety control threshold for the training equipment by combining the fitness coefficient, the movement completion rate, and the exercise risk coefficient includes: Obtain the basic equipment parameters of the training equipment, and generate a comprehensive adjustment factor for the training equipment based on the action completion rate and the exercise risk coefficient; Based on the basic equipment parameters, the comprehensive adjustment factor, and the physical fitness adaptation coefficient, the safety control threshold of the training equipment is calculated.

[0015] To address the above problems, the present invention also provides a collaborative control system for fitness videos and training equipment, the system comprising: The data synchronization acquisition module is used to collect fitness teaching video streams from the cloud to extract the skeletal key point sequence, movement intensity level and standard movement trajectory of the demonstration movements, and to collect the user's real-time physiological data and real-time movement posture through the sensor group integrated into the training equipment. The mapping and decision module is used to identify the target action of each stage in the fitness teaching video stream according to the skeletal key point sequence, and generate the target control parameters of the training device corresponding to the target action according to the action intensity level and the standard motion trajectory, so as to construct the video-parameter mapping library of the training device. The safety threshold generation module is used to determine the user's physical fitness adaptation coefficient based on the real-time physiological information and the action intensity level, calculate the user's action completion degree and exercise risk coefficient based on the real-time movement posture and the standard movement trajectory, and generate the safety control threshold of the training device by combining the physical fitness adaptation coefficient, the action completion degree and the exercise risk coefficient. The collaborative control execution module is used to generate collaborative control instructions for the training device based on the video-parameter mapping library and the security control threshold, so as to execute the collaborative control of the training device.

[0016] Compared to the problems described in the background technology, this invention, by collecting real-time physiological data and movement postures of users, combined with movement intensity levels, can determine the user's fitness fit coefficient and adjust training equipment parameters according to individual user differences to achieve personalized training programs that meet the needs of different users. This invention extracts the skeletal key point sequence, movement intensity level, and standard movement trajectory of demonstration movements from fitness videos and generates corresponding equipment control parameters, achieving intelligent matching of equipment parameters and movement rhythm, ensuring the accuracy and standardization of user movements, and improving training efficiency. This invention monitors user physiological data and movement postures in real time, calculates movement completion and movement risk coefficients, and dynamically adjusts equipment parameters and safety control thresholds based on the results, effectively preventing user sports injuries and ensuring training safety. This invention constructs a video-parameter mapping library, mapping movement features in fitness videos to equipment control parameters, achieving standardized movement guidance, helping users learn correct movement postures and avoiding injuries caused by incorrect movements. Finally, this invention can monitor the user's movement completion in real time and compare it with standard movements, promptly identifying and correcting user errors, helping users master correct movement techniques, and improving training effectiveness. Therefore, this invention can improve the adaptability and safety of collaborative control of training equipment. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a collaborative control method for fitness videos and training equipment according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the overall collaborative control process of a collaborative control method for fitness videos and training equipment provided in an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the collaborative control method for a fitness video and training equipment according to an embodiment of the present invention. Figure 4 This is a schematic diagram of a module for implementing the collaborative control method between the fitness video and the training equipment according to an embodiment of the present invention; Figure 5 A schematic diagram of a computer device for a collaborative control method of fitness videos and training equipment provided in an embodiment of the present invention; The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0018] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0019] This application provides a method for the coordinated control of fitness videos and training equipment. The executing entity of this method includes, but is not limited to, at least one electronic device that can be configured to execute the method provided in this application, such as a server or a terminal. In other words, the coordinated control method of fitness videos and training equipment can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0020] Reference Figure 1 The diagram shown is a flowchart illustrating a collaborative control method for fitness videos and training equipment according to an embodiment of the present invention. In this embodiment, the collaborative control method for fitness videos and training equipment includes: S1. Collect fitness instructional video streams from the cloud to extract the skeletal key point sequence, movement intensity level, and standard movement trajectory of the demonstrated movements, and collect the user's real-time physiological data and real-time movement posture through the sensor group integrated into the training equipment.

[0021] This invention provides data support for subsequent analysis of demonstrated movements in fitness instructional video streams collected from the cloud. The fitness instructional video stream refers to a sequence of digital video data used to guide users in training.

[0022] This invention, through extracting the skeletal key point sequence, movement intensity level, and standard motion trajectory of a demonstration movement, can convert unstructured video images into computer-processable temporal coordinate data and numerical parameters. The demonstration movement refers to a standard posture demonstrated by a professional instructor in a fitness instructional video stream. The skeletal key point sequence refers to the set of coordinate data of the center points of each joint, arranged sequentially on the timeline, used to describe the posture of the human body in the demonstration movement. The movement intensity level refers to the difficulty level classified according to the displacement amplitude, speed, and rhythmic characteristics of the demonstration movement. The standard motion trajectory refers to the smoothed ideal spatial path curve formed by key human joints during the execution of the demonstration movement.

[0023] As an embodiment of the present invention, the extraction of the skeletal key point sequence, movement intensity level, and standard movement trajectory of the demonstration movement includes: Identify the demonstration subject in each frame of the fitness instruction video stream, determine the skeletal key points of the demonstration subject, and integrate the skeletal key point sequence of the demonstration movement; Based on the skeletal key point sequence, the joint movement speed, movement amplitude, and movement acceleration of the demonstrated movement are calculated, and the repetition period of the demonstrated movement is identified in order to calculate the movement frequency of the demonstrated movement. The intensity level of the demonstration movement is determined based on the joint movement speed, the movement amplitude, the movement acceleration, and the movement frequency. The pixel coordinates of key points in the skeletal key point sequence are converted into spatial coordinates to construct the standard motion trajectory of the demonstration action.

[0024] In this context, the "demonstration subject" refers to the person demonstrating the movements in the fitness instructional video stream. The "skeletal key points" refer to characteristic points of human joint positions, such as the center points of joints like the shoulder, elbow, wrist, hip, knee, and ankle. The "joint movement speed" refers to the change in position of the skeletal key point per unit time. The "motion amplitude" refers to the maximum displacement distance of the skeletal key point in a specific direction during the movement. The "motion acceleration" refers to the change in the movement speed of the skeletal key point per unit time. The "repetition period" refers to the time required for the demonstrated movement to complete one full start and end cycle. The "movement frequency" refers to the number of times the demonstrated movement is repeated per unit time. The "key point pixel coordinates" refer to the position of the skeletal key point on the video image plane, expressed in pixels (horizontal and vertical coordinates). The "spatial coordinates" refer to the position information of the key point in three-dimensional physical space.

[0025] Optionally, the demonstration subject can be identified using object detection algorithms, such as YOLO (You Only LookOnce) or SSD (Single Shot MultiBox Detector). The skeletal key points can be determined using human pose estimation algorithms, such as MediaPipe Pose, MoveNet (Lightning / Tiger), or BlazePose.

[0026] Optionally, the joint motion velocity can be calculated using the numerical differentiation method, the motion amplitude can be calculated using the extreme difference method and the joint angle method, and the motion acceleration can be calculated using the second-order difference method.

[0027] Optionally, the action intensity level can be determined by a clustering algorithm, such as K-Means clustering.

[0028] Optionally, the standard motion trajectory can be constructed by curve fitting, such as B-spline curves, Bézier curves, or polynomial fitting.

[0029] This invention, through a sensor array integrated into the training device, collects real-time physiological data and real-time movement posture of the user, providing a data foundation for subsequent calculations of physical fitness coefficients and exercise risk coefficients. The real-time physiological data refers to vital sign signals reflecting the user's physical functional state, collected in real-time by the sensor array, including heart rate, blood oxygen saturation, respiratory rate, and electromyographic signals. The real-time movement posture refers to physical data describing the user's spatial position, limb angles, and movement patterns, collected in real-time by the sensor array, including acceleration, angular velocity, magnetic field direction, and joint angles.

[0030] As an embodiment of the present invention, the acquisition of the user's real-time physiological data and real-time motion posture through the sensor group integrated in the training device includes: Configure the multimodal sensors of the training device, construct the layout diagram and communication protocol of the multimodal sensors, and integrate the collaborative sensing network of the training device; Based on the collaborative sensing network, the user's raw physiological data and raw posture data are collected synchronously, and the raw physiological data and raw posture data are processed to obtain real-time physiological data and real-time motion posture.

[0031] The multimodal sensor refers to a sensor combination integrated into the training equipment, capable of simultaneously acquiring multiple different types of physical signals. The layout diagram refers to the topological structure diagram showing the installation positions, angles, and spatial distribution of the multimodal sensors on different components of the training equipment. The communication protocol refers to the communication specifications followed when data is transmitted between the sensors and the main control unit, or between the sensors themselves. The collaborative sensing network refers to a distributed sensing system that connects the multimodal sensors distributed throughout the training equipment into an organic whole, enabling comprehensive and seamless collaborative monitoring of the user's state. The raw physiological data refers to the initial signal values ​​directly acquired and output by the multimodal sensors, before processing, such as the light intensity and voltage values ​​read by photoelectric sensors or the raw bioelectric waveforms read by electrodes. The raw attitude data refers to the raw measurement values ​​before being processed by the attitude fusion algorithm, such as the voltage values ​​of a three-axis accelerometer or the raw angular velocity values ​​of a three-axis gyroscope.

[0032] Optionally, the layout diagram can be determined using 3D modeling software, such as SolidWorks or Blender.

[0033] Optionally, the real-time physiological data can be obtained through signal filtering and feature extraction, and the real-time motion posture can be obtained through posture calculation algorithms, such as Kalman filtering or complementary filtering algorithms.

[0034] S2. Based on the skeletal key point sequence, identify the target action for each stage in the fitness instruction video stream, and generate the target control parameters of the training device corresponding to the target action based on the action intensity level and the standard motion trajectory, so as to construct the video-parameter mapping library of the training device.

[0035] This invention, by identifying the target movements at each stage of a fitness instruction video stream based on the skeletal keypoint sequence, can provide standard time anchors for subsequent real-time movement stage identification and comparison for users. The target movement refers to a sub-movement with a specific motor function that constitutes a complete demonstration movement, defined according to the temporal characteristics of the fitness instruction video stream.

[0036] As an embodiment of the present invention, the step of identifying the target movement of each stage in the fitness instruction video stream based on the skeletal key point sequence includes: Based on the skeletal key point sequence, the motion features of the fitness instruction video stream are extracted to determine the motion boundaries of the demonstrated movements in the fitness instruction video stream. Based on the action boundaries, the fitness instruction video stream is divided into fitness action stages; Identify the target movements in each stage of the fitness exercise from the fitness instruction video stream.

[0037] The movement characteristics refer to quantitative indicators calculated based on the sequence of skeletal key points, used to describe the kinematic properties of the demonstrated movement, including the spatial coordinates of the key points, changes in joint angles, limb movement speed, and acceleration. The movement boundary refers to the critical moments used to define the complete movement cycle and different movement patterns, including the start and end times of the movement, as well as the points where adjacent movement phases switch. The fitness movement phase refers to segments with specific movement attributes that divide the continuous demonstrated movement trajectory in the time dimension according to the movement boundary, such as the warm-up phase, training phase, or rest phase between sets.

[0038] Optionally, the motion features can be obtained by kinematic feature calculation, such as first-order difference operation, second-order difference calculation, etc.

[0039] Optionally, the action boundary can be determined by a peak detection algorithm, such as the find_peaks algorithm.

[0040] This invention, through generating target control parameters for the training equipment corresponding to the target action based on the action intensity level and the standard motion trajectory, establishes an adaptive matching mechanism between the resistance output of the training equipment and the action intensity. This enables dynamic adjustment and coordinated control of the equipment parameters along the standard motion trajectory. The target control parameters refer to a set of numerical instructions used to drive and adjust the operating state of the training equipment, such as target resistance value, target power, or target load percentage.

[0041] As an embodiment of the present invention, generating target control parameters for the training device corresponding to the target action based on the action intensity level and the standard motion trajectory includes: The action intensity level is converted into a target physical quantity, and the basic load for completing the target action is determined; Extract the kinematic parameters of the standard motion trajectory, and discretize the standard motion trajectory to obtain a time-space coordinate sequence; Calculate the time-space resistance sequence of the time-space coordinate sequence based on the kinematic parameters, the target physical quantity, and the basic load; Based on the time-space resistance sequence, the target velocity curve, soft limit parameters, and hard limit parameters of the training device are constructed to generate the target control parameters of the training device.

[0042] The target physical quantity refers to a specific physical indicator characterizing the intensity of the training load, including the target power value, target torque value, and resistance percentage. The base load refers to the initial resistance value adjusted as a benchmark on the standard motion trajectory. The kinematic parameters are parameters extracted from the standard motion trajectory that describe the motion state characteristics of the object, including displacement, velocity, acceleration, and their changes over time. The time-space coordinate sequence refers to a set of ordered coordinate points containing time and position information, obtained by discretizing the continuous standard motion trajectory over time. The time-space resistance sequence refers to an ordered data set consisting of the resistance values ​​that the device needs to output, corresponding to each coordinate point in the time-space coordinate sequence. The target velocity curve refers to the desired velocity trajectory function controlling the movement of the training device's moving parts over time. The soft limit parameter refers to a safety threshold limiting the range of motion of the training device. The hard limit parameter refers to the absolute safety boundary limiting the maximum physical travel of the training device.

[0043] Optionally, the target physical quantity can be obtained by an interpolation algorithm, such as linear interpolation or polynomial interpolation.

[0044] Optionally, the time-space coordinate sequence can be obtained by an interval sampling algorithm, such as an equal time interval sampling algorithm or an equal spatial interval sampling algorithm.

[0045] Optionally, the time-space resistance sequence can be calculated by inversely solving the Newton-Euler equation or the Lagrange equation.

[0046] This invention, through the construction of a video-parameter mapping library for the training equipment, establishes a standardized correspondence between different fitness movements and equipment parameters, thereby enabling rapid conversion between video content and equipment control commands. The video-parameter mapping library refers to a storage table that maps and stores the movement features extracted from fitness instruction videos to various target control parameters required by the training equipment.

[0047] Optionally, the video-parameter mapping library of the training device can be constructed using deep learning regression, such as a multilayer perceptron or a Transformer.

[0048] S3. Based on the real-time physiological information and the intensity level of the movement, determine the user's physical fitness coefficient; based on the real-time movement posture and the standard movement trajectory, calculate the user's movement completion degree and movement risk coefficient; and combine the physical fitness coefficient, the movement completion degree, and the movement risk coefficient to generate the safety control threshold of the training device.

[0049] This invention improves the adaptability of training equipment to individuals with different fitness levels by determining the user's fitness fit coefficient based on the real-time physiological information and the movement intensity level. The fitness fit coefficient refers to the deviation ratio between the user's real-time collected physiological data and the current movement intensity level.

[0050] As an embodiment of the present invention, determining the user's fitness compatibility coefficient based on the real-time physiological information and the intensity level of the movement includes: Based on the real-time physiological information, the user's real-time physiological characteristic values ​​are determined; Based on the user's personal profile, calculate the user's target physiological characteristic value and physiological safety upper limit in the action intensity level; Based on the real-time physiological characteristic values, the target physiological characteristic values, and the physiological safety upper limit, the user's physical fitness adaptation coefficient is calculated.

[0051] The real-time physiological characteristic values ​​refer to quantitative indicators reflecting the user's current instantaneous physical state, obtained after cleaning, smoothing, and feature extraction of the collected raw physiological signals. These include real-time heart rate, real-time electromyography amplitude, and blood oxygen saturation. The target physiological characteristic value refers to the theoretically optimal physiological load value that the user should achieve, calculated by combining the user's personal profile with the intensity level of the action during the current video session. The physiological safety upper limit refers to the highest warning threshold that physiological characteristic values ​​are allowed to reach.

[0052] Optionally, the real-time physiological characteristic values ​​can be determined using a sliding window statistical method.

[0053] Optionally, the target physiological feature value can be calculated using a machine learning regression model, such as random forest regression or support vector regression (SVR).

[0054] As another implementation, the physical fitness coefficient is calculated using the following formula:

[0055] in, Indicates the fitness coefficient. Represents real-time physiological characteristic values. Represents the target physiological characteristic value. Indicates the upper limit of physiological safety. This represents the bias sensitivity coefficient. Represents the maximum value function. Indicates exercise time. This indicates that the time of motion is differentiated. This represents the rate of change over time of real-time physiological characteristics. This represents a penalty factor for fatigue trends.

[0056] It needs to be explained that in this application, in the formula, This represents the bias sensitivity coefficient, used to adjust the system's tolerance to physiological deviations. The larger the value, the more significant the decrease in the fitness coefficient will be even with slight physiological fluctuations, with a value range of [2, 10]. The fatigue trend penalty factor is used to adjust the weight of the effect of the rate of change of physiological data on the fitting coefficient, and its value range is [5, 30].

[0057] This invention, through calculation of the user's action completion rate and motion risk coefficient based on the real-time motion posture and the standard motion trajectory, enables digital quantitative evaluation and precise error correction of the user's motion posture, effectively ensuring the user's training effect and training safety. The action completion rate refers to a numerical parameter measuring the accuracy and standard of the user's current action. The motion risk coefficient refers to a numerical parameter representing the probability of the user experiencing muscle strain, joint injury, or accidents during the current exercise.

[0058] As an embodiment of the present invention, the step of calculating the user's action completion degree and motion risk coefficient based on the real-time motion posture and the standard motion trajectory includes: Extract the user's real-time key point sequence from the real-time motion posture, and extract the standard key point sequence of the standard motion trajectory; Based on the real-time keypoint sequence and the standard keypoint sequence, the real-time motion features of the real-time motion posture and the standard motion features of the standard motion trajectory are extracted respectively. Based on the real-time motion characteristics and the standard motion characteristics, the user's action completion rate and motion risk coefficient are calculated.

[0059] The real-time keypoint sequence refers to the set of skeletal keypoint coordinate data for each frame of the image during the user's movement. The standard keypoint sequence refers to the set of skeletal keypoint coordinate data arranged chronologically in the demonstrated movement. The real-time motion features refer to high-dimensional descriptive information reflecting the user's current motion state, such as keypoint angles and angular velocities. The standard motion features refer to high-dimensional descriptive information reflecting the standardized characteristics of the demonstrated movement.

[0060] As another implementation, the degree of action completion and the risk coefficient of the movement are expressed by the following formula:

[0061]

[0062] in, Indicates the degree of completion of the action. Indicates the risk factor of exercise. This indicates the degree of completion of the perfect action. This represents the total number of keypoints in the real-time keypoint sequence. Indicates An exponential function with base 0. Represents the shape sensitivity coefficient. Represents the first key point in the real-time key point sequence. Real-time coordinate vectors of key points Indicating the first key point in the standard key point sequence Standard coordinate vectors of key points Describes the Euclidean norm. Indicates the spatial scale normalization factor. Indicates static attitude risk factor. Represents the hyperbolic tangent function. Indicates the first Risk weighting coefficients for key points Represents the first in real-time motion features Real-time angles of key points The first characteristic of standard motion The standard angle of a key point Indicates the first The angle tolerance width at each key point Represents the first in real-time motion features Real-time angular velocity at key points The first characteristic of standard motion Standard angular velocity at key points Indicates the first The angular tolerance width at each key point.

[0063] It needs to be explained that in this application, in the formula, This represents the shape sensitivity coefficient, with a value range of (0,1]. The larger the value, the heavier the penalty for small posture deviations, and the faster the score drops. This represents the spatial scale normalization factor, with a value range of [0.5, 1] ​​meters, used to eliminate absolute size errors caused by varying distances from the camera. Indicates the first The risk weight coefficient for each key point ranges from (0,1). This represents the static attitude risk factor, with a value range of [0,1], which adjusts the proportion of static error angle and dynamic speed in the total risk.

[0064] Optionally, the step of extracting real-time motion features of the real-time motion posture and standard motion features of the standard motion trajectory based on the real-time keypoint sequence and the standard keypoint sequence, respectively, includes: Time-normalize the real-time keypoint sequence and the standard keypoint sequence to obtain a normalized real-time keypoint sequence and a normalized standard keypoint sequence. Calculate the Euclidean distance between the regularized real-time keypoint sequence and the regularized standard keypoint sequence to determine the posture difference between the real-time motion posture and the standard motion trajectory; Calculate the real-time dynamic parameters of the regularized real-time keypoint sequence and the standard dynamic parameters of the regularized standard keypoint sequence, respectively. By combining the posture differences, the real-time dynamic parameters, and the standard dynamic parameters, the real-time motion characteristics of the real-time motion posture and the standard motion characteristics of the standard motion trajectory are determined.

[0065] The regularized real-time keypoint sequence refers to the real-time keypoint sequence after time regularization of the real-time keypoint sequence and the standard keypoint sequence. The regularized standard keypoint sequence refers to the standard keypoint sequence after time regularization of the real-time keypoint sequence and the standard keypoint sequence. The keypoint Euclidean distance refers to the straight-line distance between the corresponding keypoint coordinates of the user's real-time keypoint sequence and the standard keypoint sequence at the same moment. The posture difference refers to the geometric conformity between the user's overall motion shape and the standard trajectory. The real-time dynamic parameters refer to the numerical values ​​describing the kinematic characteristics extracted from the regularized real-time keypoint sequence, such as joint angles and angular velocities. The standard dynamic parameters refer to the corresponding standard kinematic characteristic values ​​extracted from the regularized standard keypoint sequence, such as ideal joint angles and ideal angular velocities.

[0066] Optionally, the regularized real-time keypoint sequence and the regularized standard keypoint sequence can be obtained by a dynamic time warping algorithm, such as FastDTW (Fast Dynamic Time Warping).

[0067] Optionally, the real-time dynamic parameters can be calculated using the finite difference method, such as first-order difference, second-order difference, etc.

[0068] This invention, by combining the fitness adaptation coefficient, the movement completion rate, and the exercise risk coefficient, generates a safety control threshold for the training device. This threshold can be adjusted in real-time based on the user's physiological state and movement quality, achieving personalized and precise protection tailored to individual user differences. Specifically, the safety control threshold refers to a set of numerical boundaries that limit the maximum resistance value, minimum resistance value, maximum movement speed, and shutdown protection trigger conditions allowed by the training device within the current exercise cycle.

[0069] As an embodiment of the present invention, the step of generating a safety control threshold for the training equipment by combining the physical fitness coefficient, the movement completion rate, and the exercise risk coefficient includes: Obtain the basic equipment parameters of the training equipment, and generate a comprehensive adjustment factor for the training equipment based on the action completion rate and the exercise risk coefficient; Based on the basic equipment parameters, the comprehensive adjustment factor, and the physical fitness adaptation coefficient, the safety control threshold of the training equipment is calculated.

[0070] The basic equipment parameters refer to the theoretical equipment control parameters generated by a video-parameter mapping library based on the fitness video content during the current training phase, without any personalized adjustments. The comprehensive adjustment factor refers to a coefficient that combines the movement completion degree and the exercise risk coefficient, used to dynamically scale the basic equipment parameters.

[0071] Optionally, the integrated adjustment factor can be generated by a lightweight neural network, such as a one-dimensional convolutional network or a depthwise separable convolutional network.

[0072] S4. Generate collaborative control instructions for the training device based on the video-parameter mapping library and the security control threshold to execute collaborative control of the training device.

[0073] This invention, through the generation of collaborative control commands for the training device based on the video-parameter mapping library and the safety adjustment threshold, enables the output resistance of the training device to be automatically and in real-time adjusted according to the user's video analysis results, matching the user's current movement amplitude and force application rhythm. The collaborative control commands refer to a set of underlying control signals used to drive the motor of the training device, generated based on the ideal operating parameters of the device determined by the video-parameter mapping library and constrained by the safety adjustment threshold.

[0074] Optionally, the coordinated control command can be generated using a PID (proportional-integral-derivative) control algorithm.

[0075] The embodiments of the present invention can effectively improve the user's training effect and exercise safety by performing the coordinated control of the training device.

[0076] For a further understanding of the collaborative control of the training equipment, please refer to... Figure 2 and Figure 3 , Figure 2 This is a schematic diagram of the overall collaborative control process of a collaborative control method for fitness videos and training equipment according to an embodiment of the present invention. Steps 2-4 in the diagram involve S1 and S2 of the present invention, which extract skeletal key point sequences, action intensity levels, and standard motion trajectories from the video stream and construct a mapping library. Steps 5-7 in the diagram involve S1 and S3 of the present invention, which collect real-time physiological data and movement postures of users through the sensor group integrated in the training equipment, calculate the fitness adaptation coefficient, action completion degree, and movement risk coefficient, and fuse the fitness adaptation coefficient, action completion degree, and movement risk coefficient to generate a safety control threshold. Steps 8-9 in the diagram involve S4 of the present invention, which generates and executes collaborative control instructions based on the video-parameter mapping library and the safety control threshold.

[0077] Figure 3This is a schematic diagram illustrating the collaborative control method for a fitness video and training equipment according to an embodiment of the present invention. The diagram shows the joint control process of the user, training equipment, and cloud. The user, as the controlled object and data source, is sensed by a sensor array, generating real-time physiological data and real-time movement posture, which are then uploaded to the decision system. The cloud, as the control center and knowledge base, provides fitness instructional video streams, analyzes the videos, extracts demonstration movement features, constructs and maintains a video-parameter mapping library, and provides baseline control parameters for the equipment. The training equipment, as the controlled object and execution terminal, receives basic equipment parameters and safety control thresholds from the cloud, integrates the two to generate the final collaborative control command, and executes the collaborative control, acting on the user.

[0078] like Figure 4 The diagram shown is a functional block diagram of a collaborative control system for fitness videos and training equipment according to the present invention.

[0079] The collaborative control system 400 for fitness videos and training equipment described in this invention can be installed in an electronic device. Depending on the functions implemented, the collaborative control system includes a data synchronization acquisition module 401, a mapping and decision-making module 402, a safety threshold generation module 403, and a collaborative control execution module 404. The module described in this invention can also be called a unit, referring to a series of computer program segments that can be executed by the processor of an electronic device and perform a fixed function, stored in the memory of the electronic device.

[0080] In this embodiment of the invention, the functions of each module / unit are as follows: The data synchronization acquisition module 401 is used to acquire fitness teaching video streams from the cloud to extract the skeletal key point sequence, movement intensity level and standard movement trajectory of the demonstration movements, and to acquire the user's real-time physiological data and real-time movement posture through the sensor group integrated into the training equipment. The mapping and decision module 402 is used to identify the target action of each stage in the fitness teaching video stream according to the skeletal key point sequence, and generate the target control parameters of the training device corresponding to the target action according to the action intensity level and the standard motion trajectory, so as to construct the video-parameter mapping library of the training device. The safety threshold generation module 403 is used to determine the user's physical fitness adaptation coefficient based on the real-time physiological information and the action intensity level, calculate the user's action completion degree and exercise risk coefficient based on the real-time movement posture and the standard movement trajectory, and generate the safety control threshold of the training device by combining the physical fitness adaptation coefficient, the action completion degree and the exercise risk coefficient. The collaborative control execution module 404 is used to generate collaborative control instructions for the training device based on the video-parameter mapping library and the security control threshold, so as to execute the collaborative control of the training device.

[0081] In detail, the modules in the collaborative control system 200 for fitness videos and training equipment described in this embodiment of the invention employ the same methods as described above during use. Figure 1 The method described herein is the same as the collaborative control method for fitness videos and training equipment, and can produce the same technical effect, so it will not be elaborated here.

[0082] In one embodiment, a computer device is provided, which may be a server or a client, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a collaborative control method for fitness videos and training equipment on the server or client side.

[0083] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: The data synchronization acquisition module is used to collect fitness teaching video streams from the cloud to extract the skeletal key point sequence, movement intensity level and standard movement trajectory of the demonstration movements, and to collect the user's real-time physiological data and real-time movement posture through the sensor group integrated into the training equipment. The mapping and decision module is used to identify the target action of each stage in the fitness teaching video stream according to the skeletal key point sequence, and generate the target control parameters of the training device corresponding to the target action according to the action intensity level and the standard motion trajectory, so as to construct the video-parameter mapping library of the training device. The safety threshold generation module is used to determine the user's physical fitness adaptation coefficient based on the real-time physiological information and the action intensity level, calculate the user's action completion degree and exercise risk coefficient based on the real-time movement posture and the standard movement trajectory, and generate the safety control threshold of the training device by combining the physical fitness adaptation coefficient, the action completion degree and the exercise risk coefficient. The collaborative control execution module is used to generate collaborative control instructions for the training device based on the video-parameter mapping library and the security control threshold, so as to execute the collaborative control of the training device.

[0084] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: The data synchronization acquisition module is used to collect fitness teaching video streams from the cloud to extract the skeletal key point sequence, movement intensity level and standard movement trajectory of the demonstration movements, and to collect the user's real-time physiological data and real-time movement posture through the sensor group integrated into the training equipment. The mapping and decision module is used to identify the target action of each stage in the fitness teaching video stream according to the skeletal key point sequence, and generate the target control parameters of the training device corresponding to the target action according to the action intensity level and the standard motion trajectory, so as to construct the video-parameter mapping library of the training device. The safety threshold generation module is used to determine the user's physical fitness adaptation coefficient based on the real-time physiological information and the action intensity level, calculate the user's action completion degree and exercise risk coefficient based on the real-time movement posture and the standard movement trajectory, and generate the safety control threshold of the training device by combining the physical fitness adaptation coefficient, the action completion degree and the exercise risk coefficient. The collaborative control execution module is used to generate collaborative control instructions for the training device based on the video-parameter mapping library and the security control threshold, so as to execute the collaborative control of the training device.

[0085] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0086] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0087] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0088] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0089] Finally, it should be noted that in the above embodiments, each embodiment can be combined with each other or independent. Deleting any one of them will not affect the technical implementation of other embodiments. The above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for coordinated control of fitness videos and training equipment, characterized in that, The method includes: The system collects fitness instructional video streams from the cloud to extract the skeletal key point sequence, movement intensity level, and standard movement trajectory of the demonstrated movements. It also collects the user's real-time physiological data and real-time movement posture through the sensor group integrated into the training equipment. Based on the skeletal key point sequence, the target action of each stage in the fitness teaching video stream is identified. Based on the action intensity level and the standard motion trajectory, the target control parameters of the training device corresponding to the target action are generated to construct the video-parameter mapping library of the training device. Based on the real-time physiological information and the intensity level of the movement, the user's physical fitness coefficient is determined. Based on the real-time movement posture and the standard movement trajectory, the user's movement completion degree and movement risk coefficient are calculated. Combining the physical fitness coefficient, the movement completion degree and the movement risk coefficient, the safety control threshold of the training device is generated. Based on the video-parameter mapping library and the security control threshold, collaborative control instructions for the training device are generated to execute the collaborative control of the training device.

2. The collaborative control method for fitness videos and training equipment as described in claim 1, characterized in that, The calculation of the user's action completion rate and motion risk coefficient based on the real-time motion posture and the standard motion trajectory includes: Extract the user's real-time key point sequence from the real-time motion posture, and extract the standard key point sequence of the standard motion trajectory; based on the real-time key point sequence and the standard key point sequence, extract the real-time motion features of the real-time motion posture and the standard motion features of the standard motion trajectory, respectively. Based on the real-time motion characteristics and the standard motion characteristics, the user's action completion rate and motion risk coefficient are calculated.

3. The collaborative control method for fitness videos and training equipment as described in claim 2, characterized in that, The step of extracting real-time motion features of the real-time motion posture and standard motion features of the standard motion trajectory based on the real-time keypoint sequence and the standard keypoint sequence, respectively, includes: Time-normalize the real-time keypoint sequence and the standard keypoint sequence to obtain a normalized real-time keypoint sequence and a normalized standard keypoint sequence. Calculate the Euclidean distance between the regularized real-time keypoint sequence and the regularized standard keypoint sequence to determine the posture difference between the real-time motion posture and the standard motion trajectory; Calculate the real-time dynamic parameters of the regularized real-time keypoint sequence and the standard dynamic parameters of the regularized standard keypoint sequence, respectively. By combining the posture differences, the real-time dynamic parameters, and the standard dynamic parameters, the real-time motion characteristics of the real-time motion posture and the standard motion characteristics of the standard motion trajectory are determined.

4. The collaborative control method for fitness videos and training equipment as described in claim 1, characterized in that, The extraction of the skeletal key point sequence, movement intensity level, and standard movement trajectory of the demonstration movement includes: Identify the demonstration subject in each frame of the fitness instruction video stream, determine the skeletal key points of the demonstration subject, and integrate the skeletal key point sequence of the demonstration movement; Based on the skeletal key point sequence, the joint movement speed, movement amplitude, and movement acceleration of the demonstrated movement are calculated, and the repetition period of the demonstrated movement is identified in order to calculate the movement frequency of the demonstrated movement. Based on the joint movement speed, the movement amplitude, the movement acceleration, and the movement frequency, the movement intensity level of the demonstration movement is determined, and the pixel coordinates of key points in the skeletal key point sequence are converted into spatial coordinates to construct the standard movement trajectory of the demonstration movement.

5. The collaborative control method for fitness videos and training equipment as described in claim 1, characterized in that, The real-time physiological data and real-time motion posture of the user are collected through the sensor group integrated into the training device, including: Configure the multimodal sensors of the training device, construct the layout diagram and communication protocol of the multimodal sensors, and integrate the collaborative sensing network of the training device; Based on the collaborative sensing network, the user's raw physiological data and raw posture data are collected synchronously, and the raw physiological data and raw posture data are processed to obtain real-time physiological data and real-time motion posture.

6. The collaborative control method for fitness videos and training equipment as described in claim 1, characterized in that, The step of identifying the target movement for each stage in the fitness instruction video stream based on the skeletal key point sequence includes: Based on the skeletal key point sequence, the motion features of the fitness instruction video stream are extracted to determine the motion boundaries of the demonstrated movements in the fitness instruction video stream. Based on the action boundaries, the fitness instruction video stream is divided into fitness action stages; Identify the target movements in each stage of the fitness exercise from the fitness instruction video stream.

7. The collaborative control method for fitness videos and training equipment as described in claim 1, characterized in that, The step of generating target control parameters for the training device corresponding to the target action based on the action intensity level and the standard motion trajectory includes: The action intensity level is converted into a target physical quantity, and the basic load for completing the target action is determined. The kinematic parameters of the standard motion trajectory are extracted, and the standard motion trajectory is discretized to obtain a time-space coordinate sequence. Calculate the time-space resistance sequence of the time-space coordinate sequence based on the kinematic parameters, the target physical quantity, and the basic load; Based on the time-space resistance sequence, the target velocity curve, soft limit parameters, and hard limit parameters of the training device are constructed to generate the target control parameters of the training device.

8. The collaborative control method for fitness videos and training equipment as described in claim 1, characterized in that, The process of determining the user's physical fitness coefficient based on the real-time physiological information and the intensity level of the movement includes: Based on the real-time physiological information, the user's real-time physiological characteristic values ​​are determined; Based on the user's personal profile, the target physiological characteristic value and physiological safety limit of the user in the action intensity level are calculated. Based on the real-time physiological characteristic value, the target physiological characteristic value and the physiological safety limit, the user's physical fitness adaptation coefficient is calculated.

9. The collaborative control method for fitness videos and training equipment as described in claim 1, characterized in that, The step of combining the physical fitness coefficient, the movement completion rate, and the exercise risk coefficient to generate the safety control threshold for the training equipment includes: Obtain the basic equipment parameters of the training equipment, and generate a comprehensive adjustment factor for the training equipment based on the action completion rate and the exercise risk coefficient; Based on the basic equipment parameters, the comprehensive adjustment factor, and the physical fitness adaptation coefficient, the safety control threshold of the training equipment is calculated.

10. A collaborative control system for fitness videos and training equipment, characterized in that, The system includes: The data synchronization acquisition module is used to collect fitness teaching video streams from the cloud to extract the skeletal key point sequence, movement intensity level and standard movement trajectory of the demonstration movements, and to collect the user's real-time physiological data and real-time movement posture through the sensor group integrated into the training equipment. The mapping and decision module is used to identify the target action of each stage in the fitness teaching video stream according to the skeletal key point sequence, and generate the target control parameters of the training device corresponding to the target action according to the action intensity level and the standard motion trajectory, so as to construct the video-parameter mapping library of the training device. The safety threshold generation module is used to determine the user's physical fitness adaptation coefficient based on the real-time physiological information and the action intensity level, calculate the user's action completion degree and exercise risk coefficient based on the real-time movement posture and the standard movement trajectory, and generate the safety control threshold of the training device by combining the physical fitness adaptation coefficient, the action completion degree and the exercise risk coefficient. The collaborative control execution module is used to generate collaborative control instructions for the training device based on the video-parameter mapping library and the security control threshold, so as to execute the collaborative control of the training device.

Images