A robot action training method and system based on human motion video

Abstract

The application discloses a robot action training method and system based on human motion video, which comprises the following steps: firstly, pre-processing the human motion video, extracting and enhancing the video frames; then, detecting the motion key points two-dimensional coordinates, mapping them into three-dimensional space relative coordinates, and calculating the joint rotation angle; then, constructing and optimizing the motion time sequence, and fitting the reward function; finally, training based on the reinforcement learning algorithm to obtain the target action model, and the system core relies on the deep learning network containing ResNet backbone layer and the DDPG / PPO reinforcement learning framework. The application solves the existing pain points based on a complete technical process, realizes the precise action imitation of the robot through rigorous steps, and the scheme is reproducible and extensible, and is suitable for the action training requirements of robots in multiple fields such as industrial cooperation, service and rehabilitation.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of robot motion training and computer vision, in particular to a robot motion training method and system based on human motion video, which is suitable for robot application scenarios that need to achieve precise motion control by imitating human motion, such as industrial collaborative robots, service robots, rehabilitation robots, etc. BACKGROUND

[0002] With the rapid development of robot technology, robots are increasingly widely used in industrial production, daily life, medical rehabilitation, etc. In many scenarios, robots need to be able to imitate human actions to perform specific tasks, such as industrial collaborative robots imitating the assembly actions of workers, service robots imitating the grasping actions of humans, rehabilitation robots imitating the auxiliary training actions of therapists, etc.

[0003] Currently, robot motion training mainly has the following technical problems:

[0004] Insufficient accuracy of human motion feature extraction: Traditional human motion capture methods rely mainly on dedicated motion capture devices (such as optical motion capture systems, inertial motion capture devices), which are costly and complex to operate, and cannot be used in scenarios without dedicated devices. Although video-based motion capture methods do not require dedicated devices, the pre-processing of videos in existing technologies is not perfect and is easily affected by factors such as background interference and changes in lighting, resulting in low accuracy of human key point detection and thus affecting the accuracy of subsequent motion feature extraction.

[0005] Poor universality of motion representation: Existing technologies often directly use the coordinate information of human key points as motion representation, but different individuals have different body proportions, and the absolute coordinates of key points corresponding to the same motion are different, which makes the robot model trained based on the motion of one individual unable to adapt to the motion features of other individuals, and the universality is poor. At the same time, key point coordinate information cannot be directly mapped to robot motor control parameters, and an additional conversion process is needed, increasing the technical complexity and error.

[0006] Low quality of motion time series: Motion data extracted based on video often contains dirty data due to detection errors and video noise, and motion data is prone to jitter in the time dimension, resulting in poor smoothness of motion time series. If such data is directly used for robot training, it will result in unstable robot motion execution and even failure.

[0007] Unreasonable design of reinforcement learning reward function: In existing robot reinforcement learning training, the reward function is often designed based on experience, which cannot accurately match the features of human motion, resulting in large deviation between the motion of the trained robot and the target human motion, low training efficiency, and slow convergence speed.

[0008] Action model scheduling convenience is poor: the trained robot action model is usually bound to a specific robot hardware or software system, and is difficult to be reused in different robot platforms. In addition, the flexibility and convenience of action scheduling are insufficient when the robot performs specific tasks. SUMMARY

[0009] To solve the problems of insufficient action feature extraction accuracy, poor action representation universality, low motion time sequence quality, unreasonable reward function design, and poor action model scheduling convenience in the prior art, the present application provides a robot action training method and system based on human motion video, which can accurately extract action features, realize universal action representation, improve motion data quality, optimize reinforcement learning reward function design, and realize standardized scheduling of action models based on ordinary human motion video.

[0010] According to an aspect of the present application, a robot action training method based on human motion video is provided, comprising:

[0011] Obtain a human motion video and preprocess it;

[0012] Detect the frames of the preprocessed human motion video using key point capture technology, and output the two-dimensional image coordinates of each key point in the video;

[0013] Input the two-dimensional image coordinates of each key point into a deep learning coordinate mapping model, and output the three-dimensional spatial relative coordinates of each key point. The three-dimensional spatial relative coordinates only reflect the relative positional relationship between the key points of the human body;

[0014] Based on the output three-dimensional spatial relative coordinates of each key point, obtain the rotation angle information of each joint node through spatial geometric calculation;

[0015] Arrange the obtained rotation angle information of each joint node in the order of the video time axis, construct a motion time sequence, and perform post-processing;

[0016] Use a function fitting algorithm to fit the scatter data corresponding to the post-processed motion time sequence into a reward function. The reward function is used for reward value calculation in the robot reinforcement learning training process;

[0017] Based on the obtained reward function, the robot is trained through reinforcement learning to obtain a target action model suitable for the robot to execute.

[0018] As a further technical solution, the preprocessing includes video frame extraction, human target detection and segmentation, and image enhancement.

[0019] As a further technical solution, the key point capture technology adopts a deep learning network based on skeleton key point detection, which includes a feature extraction backbone layer, a key point prediction branch layer, and a coordinate regression layer.

[0020] The feature extraction backbone layer adopts a residual neural network structure to extract deep semantic features of human targets in video frames.

[0021] The key point prediction branch layer adopts a structure combining convolutional layers and deconvolutional layers to upsample and refine the deep semantic features, outputting a key point heat map.

[0022] The coordinate regression layer is based on the key point heat map, and the center positions of each key point are located by a peak detection algorithm, and then the two-dimensional image coordinates of the key points are calculated. The key points at least include the core motion-related key points of the head vertex, neck, shoulder, elbow, wrist, hip, knee, and ankle.

[0023] As a further technical solution, the deep learning coordinate mapping model is an end-to-end three-dimensional coordinate regression network, including an input layer, a feature fusion layer, and a three-dimensional coordinate output layer.

[0024] The input layer receives the two-dimensional image coordinates of the key points and the corresponding video frame context features, which are extracted from the video frame by a convolutional neural network.

[0025] The feature fusion layer adopts a structure combining attention mechanisms and fully connected layers to fuse the two-dimensional image coordinate features and the video frame context features, highlighting the key feature information for three-dimensional coordinate mapping.

[0026] The three-dimensional coordinate output layer is implemented by a multilayer perceptron, outputting the three-dimensional spatial relative coordinates of each key point. The three-dimensional spatial relative coordinates take the hip key point of the human body as the origin to establish a local coordinate system, with the X-axis along the left-right direction of the human body, the Y-axis along the up-down direction of the human body, and the Z-axis along the front-back direction of the human body.

[0027] As a further technical solution, the calculation of the rotation angle information of the joint nodes is based on the spatial vector angle formula, including:

[0028] According to the three-dimensional spatial relative coordinates of the key points, the parent key points and child key points corresponding to each joint node are determined, and a joint node-key point association relationship is established.

[0029] The vector from the parent key point corresponding to each joint node to the joint node is calculated , and the vector from the joint node to the child key point is calculated .

[0030] The spatial vector dot product formula is used to calculate the vector and The included angle is the rotation angle of the joint.

[0031] As a further technical solution, the function fitting algorithm adopts Gaussian process regression algorithm or multinomial regression algorithm, and the input of the reward function is the joint rotation angle deviation value corresponding to the current action state of the robot, and the output is the corresponding reward value.

[0032] As a further technical solution, the method also includes:

[0033] When using the Gaussian process regression algorithm, the regression model parameters are determined by maximizing the posterior probability estimate, and the motion time series scatter data is used as training samples to fit a continuous reward function curve.

[0034] When using the polynomial regression algorithm, the polynomial coefficients are solved by the least squares method, and the degree of the polynomial is determined according to the motion complexity of the motion time series.

[0035] As a further technical solution, the post-processing includes dirty data removal and motion smoothing. The dirty data removal is achieved through an outlier detection algorithm, and the motion smoothing is completed through a time-series data filtering algorithm.

[0036] As a further technical solution, the reinforcement learning training employs a deep deterministic policy gradient algorithm or a proximal policy optimization algorithm, including:

[0037] Construct the robot's motion space and state space. The motion space is the feasible domain of the rotation angle of each joint, and the state space is the deviation between the robot's current joint rotation angle and the target motion joint rotation angle.

[0038] The obtained reward function is integrated into the reward mechanism of reinforcement learning. When the deviation between the robot's action state and the target action state decreases, the reward function outputs a positive reward value, and when the deviation increases, it outputs a negative reward value.

[0039] Initialize the parameters of the reinforcement learning agent and set the hyperparameters;

[0040] During training, the agent interacts with the environment, selects actions based on the current state, obtains reward values ​​through a reward function, and updates the agent's policy network parameters using the gradient descent algorithm until training converges.

[0041] According to one aspect of the present invention, a robot motion training system based on human motion videos is provided, comprising:

[0042] The video preprocessing module is used to acquire and preprocess videos of human movement.

[0043] A key point detection module is configured to detect the preprocessed human motion video frame by using a key point capturing technology, and output two-dimensional image coordinates of each key point in the video.

[0044] A three-dimensional coordinate mapping module is configured to input the two-dimensional image coordinates of each key point into a deep learning coordinate mapping model, and output three-dimensional spatial relative coordinates of each key point, wherein the three-dimensional spatial relative coordinates only reflect the relative position relationship between the key points of the human body.

[0045] A joint angle calculation module is configured to obtain rotation angle information of each joint node by spatial geometric calculation based on the output three-dimensional spatial relative coordinates of each key point.

[0046] A time sequence construction and post-processing module is configured to arrange the obtained rotation angle information of each joint node in the order of the video time axis, construct a motion time sequence, and perform post-processing.

[0047] A reward function fitting module is configured to fit the scattered point data corresponding to the post-processed motion time sequence into a reward function by using a function fitting algorithm, wherein the reward function is used for reward value calculation in the robot reinforcement learning training process.

[0048] A reinforcement learning training module is configured to perform reinforcement learning training on the robot based on the obtained reward function, and obtain a target action model suitable for the robot to perform.

[0049] Compared with the prior art, the present application has the following advantages:

[0050] The present application solves the problems of low precision, poor universality, and difficult scheduling in the existing robot action training by using the complete process of "video preprocessing-key point detection-three-dimensional mapping-joint calculation-time sequence processing-reward fitting-reinforcement training", and the technical solution is reproducible and extensible, and is suitable for robot action imitation requirements in multiple fields. BRIEF DESCRIPTION OF DRAWINGS

[0051] To make the technical solutions in the present application or prior art clearer, the following will briefly introduce the drawings used in the embodiments or prior art description. Obviously, the drawings described below are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort.

[0052] Figure 1 A flowchart of a robot action training method based on human motion video provided by an embodiment of the present application.

[0053] Figure 2 A video preprocessing flowchart provided by an embodiment of the present application.

[0054] Figure 3 This is a schematic diagram of the key point detection process provided in an embodiment of the present invention.

[0055] Figure 4 This is a schematic diagram of the three-dimensional mapping process provided in an embodiment of the present invention.

[0056] Figure 5 This is a schematic diagram of the joint calculation process provided in an embodiment of the present invention.

[0057] Figure 6 This is a schematic diagram of the timing processing and reward fitting process provided in an embodiment of the present invention.

[0058] Figure 7 This is a schematic diagram of the reinforcement training process provided in an embodiment of the present invention. Detailed Implementation

[0059] To address the problems of insufficient accuracy in motion feature extraction, poor universality of motion representation, low quality of motion time series, unreasonable reward function design, and poor convenience of motion model scheduling in existing robot motion training and scheduling technologies, this invention provides a robot motion training method based on human motion videos, comprising the following steps:

[0060] Step 1: Human motion video preprocessing.

[0061] Acquire motion videos of humans performing target actions (captured by ordinary cameras, smartphones, etc.), and preprocess the videos to highlight the target area, providing high-quality input for subsequent keypoint recognition. The preprocessing process includes three sub-steps:

[0062] Video frame extraction: Video frames are extracted from the video using an equal-interval sampling method. The sampling frequency is determined based on the speed of human movement. For example, for fast movements (such as waving or grabbing), the sampling frequency is set to 30fps-60fps; for slow movements (such as rehabilitation training or assembly), the sampling frequency is set to 10fps-20fps, ensuring that every detail of the movement is captured completely and avoiding loss of movement information.

[0063] Human Target Detection and Segmentation: A deep learning-based target detection network is used to process the extracted video frames. The network input is the original video frame. Through feature extraction and target localization, the output is an image of the foreground region containing only human targets, eliminating background interference. In this process, a pre-trained human detection model can achieve accurate segmentation of human targets under different poses, clothing, and lighting conditions.

[0064] Image enhancement: Enhancing the segmented human target image, including:

[0065] Gray-level normalization: A linear stretching algorithm is used to map image pixel values ​​from the original range to the [0, 255] interval. The formula is as follows:

[0066] ,

[0067] Where I is the original pixel value. These represent the minimum and maximum pixel values ​​of the image, respectively, to eliminate the impact of differences in illumination intensity on subsequent detection.

[0068] Adaptive contrast adjustment: Based on histogram equalization algorithm, the grayscale histogram of the image is corrected to enhance the contours and details of human targets in the image. For example, for images with low lighting, histogram equalization can improve the clarity of dark details.

[0069] Noise suppression: A hybrid filtering algorithm combining Gaussian filtering and median filtering is adopted. First, Gaussian noise (such as image sensor noise) is suppressed by Gaussian filtering, and then salt-and-pepper noise (such as interference noise in video transmission) is eliminated by median filtering to ensure that the image quality meets the requirements of key point detection.

[0070] Step 2: Detection of 2D image coordinates of human key points.

[0071] A deep learning network based on skeletal keypoint detection is used to process preprocessed video frames, detecting and outputting the two-dimensional image coordinates of various human keypoints in the video. This deep learning network includes a feature extraction backbone layer, a keypoint prediction branch layer, and a coordinate regression layer.

[0072] Feature extraction backbone layer: The ResNet residual neural network structure is adopted to solve the gradient vanishing problem in deep network training through residual connections, effectively extracting deep semantic features of human targets in video frames (such as skeletal contours and joint position features).

[0073] Keypoint prediction branch layer: This layer employs a combination of convolutional layers (3×3 convolutional kernels) and deconvolutional layers (2×2 deconvolutional kernels) to upsample deep semantic features, restoring the feature map size to match the original video frame. Multiple convolutional layers are then used to refine the features, ultimately outputting a heatmap of each keypoint. In the heatmap, the value of each pixel represents the probability that the location is the corresponding keypoint; the higher the probability, the more likely the location is to be the center of the keypoint.

[0074] Coordinate Regression Layer: Based on the keypoint heatmap, the center position of each keypoint is located using a non-maximum suppression algorithm. Specifically, in the heatmap, the pixel coordinates corresponding to the local maximum value are found, and these coordinates are the pixel coordinates of the keypoint in the two-dimensional image. In this invention, the detected keypoints include at least 16 core motion-related keypoints: head apex, neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, right ankle, left palm, and right palm, ensuring a complete representation of the skeletal structure of human movement.

[0075] Step 3: Mapping the relative coordinates of key points in 3D space.

[0076] An end-to-end deep learning coordinate mapping model is constructed to map the 2D image coordinates of keypoints obtained in step 2 to the 3D relative coordinates of each keypoint. These coordinates only reflect the relative positional relationships between human keypoints, eliminating the interference of absolute spatial position on action representation. The deep learning coordinate mapping model includes an input layer, a feature fusion layer, and a 3D coordinate output layer.

[0077] Input Layer: This layer receives two input data sets: first, the 2D image coordinates of the 16 keypoints output from step 2 (each keypoint is represented by (x,y), totaling 32 features); second, video frame context features, extracted from the preprocessed video frames using the lightweight convolutional neural network MobileNet, containing contextual information such as texture and pose of human targets, totaling 256 features. The input layer concatenates the two input data sets into a 288-dimensional feature vector, which is then fed into the feature fusion layer.

[0078] Feature fusion layer: It adopts a structure combining self-attention mechanism and fully connected layer. The attention mechanism highlights the key features for 3D coordinate mapping and suppresses the interference of irrelevant features by calculating the importance weight of each dimension in the feature vector; the fully connected layer (containing 2 hidden layers with 512 and 256 neurons respectively) performs non-linear transformation on the attention-weighted features to achieve deep fusion of 2D coordinate features and context features.

[0079] 3D Coordinate Output Layer: 3D coordinate regression is achieved using a multilayer perceptron (MLP, containing one hidden layer with 128 neurons and an output layer with 48 neurons). The output layer outputs the 3D spatial coordinates of 16 keypoints (each keypoint is represented by (x, y, z), for a total of 48 dimensions). To eliminate interference from absolute spatial positions, a local coordinate system is established with the midpoint of the line connecting the keypoints of the left and right hips as the origin: the X-axis runs along the left-right direction of the body (positive to the right), the Y-axis runs along the up-down direction of the body (positive upwards), and the Z-axis runs along the front-back direction of the body (positive forwards). This ultimately yields the relative 3D spatial coordinates of each keypoint.

[0080] Step 4: Calculate the rotation angle of the joint.

[0081] Based on the three-dimensional relative coordinates of the key points output in step 3, the rotation angle information of each joint is obtained through spatial geometric calculation. This rotation angle information is applicable to the universal representation of the motion characteristics of human movements with different body proportions (the joint rotation angles are consistent for the same movement in different individuals), and can also be directly adapted to the motion control parameter requirements of the robot motors (the control quantity of the robot joint motors is the rotation angle). The specific calculation steps are as follows:

[0082] Establish the key-joint relationship: Based on the human skeletal structure, determine the parent and child key points corresponding to each key point. For example, the parent key point of the elbow key point (left elbow, right elbow) is the shoulder key point (left shoulder, right shoulder), and the child key point is the wrist key point (left wrist, right wrist); the parent key point of the knee key point (left knee, right knee) is the hip key point (left hip, right hip), and the child key point is the ankle key point (left ankle, right ankle); the parent key point of the neck key point is the top of the head, and the child key point is the midpoint of the line connecting the shoulders.

[0083] Calculate the keypoint correlation vector: Let keypoint J be the parent keypoint P and child keypoint C, and their relative coordinates in 3D space be respectively... .

[0084] Calculate the vector from the parent keypoint to the joint:

[0085] ,

[0086] Vector from keypoint to subkeypoint:

[0087] .

[0088] Calculate the joint rotation angle: Use the dot product formula of spatial vectors to calculate the vector. and The included angle is the rotation angle of the joint. The calculation formula is:

[0089] ,

[0090] in, , is a vector The amount, For vectors The amount, This formula can be used to calculate the rotation angles of all joints, including the elbow, knee, neck, and shoulder. The angle unit is radians (rad), which can be converted to degrees (°) according to the robot motor requirements.

[0091] Step 5: Motion time series construction.

[0092] Arrange the rotation angle information of each joint point obtained in step 4 according to the video timeline to construct a motion time series. For example, if the video frame extraction frequency is 30fps and the number of extracted video frames is 300 (corresponding to 10 seconds of action), then each joint point corresponds to 300 rotation angle data points, arranged in chronological order (frame numbers 1 to 300) to form the time series of that joint point. The combination of the time series of all joint points constitutes the complete motion time series, which can be represented as follows: Where k is the number of joints. (n is the number of video frames, (where is the rotation angle of the i-th joint point in the j-th frame).

[0093] Step 6: Post-processing of motion time series.

[0094] Post-processing is performed on the motion time series constructed in step 5, including dirty data removal and motion smoothing, to improve the quality of motion data.

[0095] Dirty data removal: using a method based on Outlier detection algorithm based on principles. For the time series of each key point. Calculate its mean with standard deviation It will exceed the range. Rotation angle data Data deemed "dirty" (such as abnormally large or small angles due to keypoint detection errors) is removed from the time series. For any missing values ​​remaining after removing dirty data, linear interpolation is used to fill in the gaps, ensuring the continuity of the time series.

[0096] Motion smoothing: A sliding window filtering algorithm is used to smooth the motion time series after removing dirty data. The sliding window size is set to N (N is an odd number, ranging from 3 to 11, adjusted according to the motion smoothing requirements; the more complex the motion, the smaller the window size). For each data point in the time series... Take N consecutive data points before and after the given data point (if the data point is at the beginning or end of the sequence and there are fewer than N data points, fill the gaps by repeating boundary data), calculate the mean of these N data points, and use this mean as the smoothed data point. The formula is:

[0097] ,

[0098] in, This is the value after boundary data processing. This motion smoothing process eliminates jitter in the time series, making the motion data smoother in the time dimension, consistent with the physical characteristics of robot motion execution.

[0099] Step 7: Reward function fitting.

[0100] A function fitting algorithm is used to fit the scatter data corresponding to the motion time series after post-processing in step 6 into a reward function. This reward function is used to calculate the reward value during the robot's reinforcement learning training process, guiding the robot's actions to converge towards the target human action. The specific process is as follows:

[0101] Determine the fitted data samples: Using the joint rotation angle deviation value corresponding to the robot's current action state as the input feature, and "similarity between the robot's action and the target human action" as the output label, construct the fitted data samples. Among them, the joint rotation angle deviation value... Let be the rotation angle of the i-th joint of the robot at time t. Let be the rotation angle of the i-th joint of the human at time t (from the post-processed motion time series); the output label (reward value) is set according to the deviation value, the smaller the deviation value, the larger the reward value, for example, using... ( To adjust the coefficients (set according to training needs), multiple sets are constructed using initial labels. Scattered data sample.

[0102] Choose a function fitting algorithm: Select either Gaussian process regression or multinomial regression algorithm based on the complexity of the operation.

[0103] ① Gaussian Process Regression Algorithm: Suitable for scenarios with high action complexity and strong nonlinearity of scattered data. By defining a covariance function, a Gaussian process model is established. Using scattered data samples as the training set, the model parameters are determined by maximizing the posterior probability estimate, and finally, a continuous reward function curve is obtained through fitting. This algorithm has good nonlinear fitting ability and can output prediction uncertainty, facilitating subsequent adjustment of the reward function.

[0104] ② Multinomial Regression Algorithm: Suitable for scenarios with low action complexity and approximately linear scatter data. Assume the reward function is... (m is the degree of the polynomial, ranging from 3 to 7), the polynomial coefficients are solved using the least squares method. This minimizes the mean square error between the fitted curve and the scatter data. For example, when m = 5, this is achieved by solving the normal equation system. ( To design the matrix, For reward value label vectors, (as coefficient vector), to obtain the coefficient vector Determine the reward function.

[0105] Step 8: Robot reinforcement learning training.

[0106] Based on the reward function obtained in step 7, the robot is trained using reinforcement learning to obtain a target action model adapted to the robot's execution. This invention employs either the Deep Deterministic Policy Gradient (DDPG) algorithm or the Proximal Policy Optimization (PPO) algorithm, with the specific steps as follows:

[0107] 1. Construct the state space and action space:

[0108] State space: defined as a vector of deviations between the robot's current joint rotation angles and the joint rotation angles corresponding to the human target action, i.e. (k is the number of key points), and the state space dimension is k;

[0109] Motion space: defined as the vector of rotation angle adjustment at each joint of the robot, i.e. (k is the number of joints, The adjustment amount is the rotation angle adjustment for the i-th joint. The range of the adjustment amount is determined based on the physical limitations of the robot motor (e.g., ...). (The specific value is set by the robot's hardware parameters), and the motion space dimension is k.

[0110] 2. Integrating the reward function and setting the training mechanism:

[0111] The reward function obtained in step 7 is integrated into the reward mechanism of reinforcement learning to form the immediate reward calculation logic: when the robot performs action A, the state transitions from S to... According to the new status Corresponding joint rotation angle deviation value The immediate reward value r is calculated using a reward function. If... If the robot's actions decrease (to make them more similar to the target human actions), then r is a positive reward (e.g.) );like If the robot's actions deviate from the target human actions, then r becomes a negative reward (e.g., );like If r is 0 (the robot's actions perfectly match the human target actions), then r is the maximum reward value.

[0112] Simultaneously set discount factors (Value range is 0.9-0.99), used to calculate the present value of future rewards, guiding the robot to focus on long-term action optimization and avoid short-term local optima.

[0113] 3. Initialize training parameters and agent:

[0114] Initialize the robot state to the zero state of each joint. =Initial deviation value), start the training loop;

[0115] If the DDPG algorithm is used: Initialize the Actor network (policy network) and the Critic network (value network). The Actor network takes state S as input and outputs action A; the Critic network takes state S and action A as input and outputs action value. Set the experience replay pool size to [value]. Target network update rate (The range of values ​​is) Learning rate .

[0116] If the PPO algorithm is used: initialize the policy network and value network, both sharing the first half of the feature extraction layer; set the Clip coefficient. (Value range is 0.1-0.2), used to limit the policy update magnitude; learning rate The batch size B is 64-256, and the number of iterations per round K is 3-10.

[0117] 4. Training loop execution and convergence determination:

[0118] After training converges, the final policy network parameters are saved to form a target action model adapted to the robot. This model can output the optimal action command based on the robot's real-time state.

[0119] DDPG algorithm training process:

[0120] S1, the agent is in the current state Next, output actions through the Actor network. And add Gaussian noise (exploration rate) Decrease with training rounds, initial =0.1, final =0.01), enhancing exploration capabilities;

[0121] S2, the robot performs the action. , transition to the new state Calculate immediate reward using reward function ;

[0122] S3, Experience Group Store in the experience replay pool;

[0123] S4, when the amount of data in the experience replay pool reaches a threshold (e.g.) When randomly sampling batches from the pool, experience is used. Update the Critic network: minimize the loss function ( For the target Critic network output, (Output of the target Actor network).

[0124] S5 updates the Actor network via policy gradient ascent: maximizing the objective function. .

[0125] S6, Update target network parameters: .

[0126] Repeat steps S1-S6 until the average reward value fluctuation of 100 consecutive training rounds is less than 5%, at which point the training is considered converged.

[0127] PPO algorithm training process:

[0128] S10, the agent is in the current state Below, the action probability distribution is output through the policy network, and actions are sampled. And execute;

[0129] S20, records trajectory data This continues until T = 2048 - 4096 trajectory data points are collected;

[0130] S30, Calculate the dominance function ( The state value output by the value network is then standardized using the advantage function.

[0131] S40 updates the policy network based on trajectory data using the Clip loss function: It is the probability ratio. (The standardized dominance function);

[0132] S50, Update the value network: Minimize the mean squared error loss ( (The target value is calculated using the time-difference method).

[0133] Repeat steps S10-S50 until the KL divergence of the policy network (the difference from the initial policy) is less than 0.01 and the average reward value is stable, then the training is considered to have converged.

[0134] To make the technical solution of this invention clearer and easier to understand, and to facilitate understanding and implementation by those skilled in the art, the following detailed description of the specific implementation process of this invention is based on the typical application scenario of upper limb lifting training movement imitation of a rehabilitation robot, and in conjunction with the accompanying drawings (the accompanying drawings in this embodiment correspond one-to-one with the technical solution of the invention, including method flowcharts, system module interaction diagrams, reinforcement learning training flowcharts, etc.). In this embodiment, the rehabilitation robot needs to imitate the "upper limb lifting movement" of a rehabilitation therapist (slowly lifting from a 90° elbow bend to a 180° elbow extension) to assist patients with limb dysfunction in rehabilitation training. The hardware equipment, software environment, and parameter settings used are all tailored to actual application needs, ensuring the reproducibility and practicality of the technical solution.

[0135] The overall process of the robot motion training method based on human motion videos provided in this embodiment is as follows: Figure 1 As shown, the specific steps are explained below:

[0136] Step 1: Human motion video preprocessing.

[0137] This step aims to eliminate background interference and noise in the video, highlight the upper limb target area, and provide high-quality input for subsequent keypoint detection, such as... Figure 2 As shown, the following sub-steps are specifically executed:

[0138] ① Video frame extraction:

[0139] Using an equal-interval sampling method, the video of the therapist's "upper limb lifting movement" (10 seconds long, 30fps) was read. 300 video frames were extracted at a sampling frequency of 30fps and stored in PNG format. The frame naming rule is "frame_001.png" to "frame_300.png" to ensure that every movement detail from elbow flexion to extension is fully captured (e.g., the first frame is the initial state of elbow flexion at 90°, and the 300th frame is the final state of elbow extension at 180°).

[0140] ② Human target detection and segmentation:

[0141] Load the pre-trained model and process each video frame:

[0142] Input: Raw video frames (1920×1080×3);

[0143] Processing procedure: The model identifies human targets in frames through feature extraction and target localization, and outputs the bounding box coordinates of the upper limb region (e.g., the upper limb bounding box in frame 1 is...). );

[0144] Output: A foreground image (450×620 resolution) containing only the upper limb target, cropped based on bounding boxes. Background areas are removed by setting pixel values ​​to 0, effectively avoiding interference from background elements such as furniture and training equipment on subsequent detection.

[0145] ③ Image enhancement:

[0146] The cropped upper limb foreground image undergoes a three-step enhancement process to ensure image quality meets the requirements for keypoint detection:

[0147] Gray-scale normalization: A linear stretching algorithm is used, based on the formula... Map image pixel values ​​from the original range (0-255, which may be locally darker or brighter due to differences in lighting) to... Standard range. For example, the range of original pixel values ​​for a given frame of an image is... Then the normalized pixel value is calculated as This eliminates the problem of uneven pixel value distribution caused by differences in light intensity;

[0148] Adaptive contrast adjustment: Based on a histogram equalization algorithm, the histogram of the gray-level normalized image is corrected. For example, for images under low-light conditions (where details in dark areas are blurred), equalization expands the gray-level histogram from being concentrated in the low gray-level range (0-80) to the full range (0-255), improving the contour clarity of key areas such as elbows and wrists, with an average contrast improvement of 30%.

[0149] Noise suppression: A hybrid filtering algorithm combining Gaussian filtering and median filtering is employed.

[0150] First, apply a 5×5 Gaussian filter ( To suppress image sensor noise (Gaussian noise), the formula is as follows: Smooth image texture;

[0151] Next, a 3×3 median filter is used to eliminate salt-and-pepper noise (random black and white noise) from the video transmission process. By taking the median of pixel values ​​within the sorting window, the edge features of key points are preserved. The enhanced image is stored for key point detection in step 2.

[0152] Step 2: Detection of 2D image coordinates of human key points.

[0153] This step employs a deep learning network based on skeletal keypoint detection to accurately locate the two-dimensional image coordinates of eight core key points of the upper limb, such as... Figure 3 As shown, the specific execution process is as follows:

[0154] ① Network structure loading and parameter settings:

[0155] Load a lightweight keypoint detection model, which consists of three core layers:

[0156] Feature extraction backbone layer: ResNet50 structure (containing 49 convolutional layers and 16 residual blocks), which solves the gradient vanishing problem in deep network training through residual connections, effectively extracting deep semantic features (such as skeletal contours and joint position textures) of upper limb images.

[0157] Keypoint prediction branch layer: Composed of three 3×3 convolutional layers (stride 1, padding 1) and two 2×2 deconvolutional layers (stride 2), it upsamples the low-resolution feature map (28×39×2048) output by the backbone layer to restore it to the same resolution as the input image (450×620), and refines the features through multiple convolutions, finally outputting a heatmap of 8 keypoints (each heatmap resolution 450×620, the pixel value represents the probability that the position is the corresponding keypoint, ranging from 0 to 1);

[0158] Coordinate regression layer: The non-maximum suppression (NMS) algorithm is used to perform peak detection on the heatmap of each key point: traverse the pixels of the heatmap, and when a pixel value is the maximum value in the 3×3 neighborhood and is greater than the probability threshold (0.8), it is determined to be the center of the key point and its pixel coordinates (x,y) are recorded.

[0159] ② Keypoint Detection and Output: Input the 300 enhanced upper limb images from step 1 into the model sequentially, and output the two-dimensional image coordinates of 8 keypoints (left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left palm, right palm). For example:

[0160] Frame 1 (elbow bent at 90°): Right shoulder coordinates (225, 120), right elbow coordinates (220, 350), right wrist coordinates (215, 580) (unit: pixels);

[0161] Frame 300 (elbow extended 180°): Right shoulder coordinates (225, 120), right elbow coordinates (220, 320), right wrist coordinates (215, 180) (unit: pixels). Keypoint coordinates for all frames are stored in a CSV file in the format "frame number, left shoulder x, left shoulder y, right shoulder x, right shoulder y, ..., right palm x, right palm y", providing input for the 3D coordinate mapping in step 3.

[0162] Step 3: Mapping the relative coordinates of key points in 3D space.

[0163] This step uses an end-to-end deep learning coordinate mapping model to convert two-dimensional pixel coordinates into three-dimensional coordinates reflecting the relative positions of key points in the upper limb, eliminating interference from absolute spatial positions. Figure 4 As shown, the specific execution process is as follows:

[0164] ① 3D coordinate mapping model construction and training: The model adopts an end-to-end structure of "input layer - feature fusion layer - 3D coordinate output layer", and the specific design is as follows:

[0165] Input layer: Receives two types of data and concatenates them into a 288-dimensional feature vector.

[0166] Step 2 outputs the 2D coordinates of 8 key points (each key point contains two pixel values, x and y, for a total of 16 features);

[0167] Video frame context features: These are extracted from the enhanced upper limb image in Step 1 using the lightweight MobileNet network (containing 28 convolutional layers and only 4.2M parameters), outputting 256-dimensional features (including contextual information such as clothing texture and joint contours) to ensure mapping accuracy.

[0168] Feature Fusion Layer: Self-Attention Mechanism: Scaled Dot-Product Attention calculates the importance weights of each dimension of the 288-dimensional feature vector, highlighting the feature contributions of key movement points such as the right shoulder, right elbow, and right wrist (increasing their weight ratio to over 60%), while suppressing interference from irrelevant features such as the left upper limb (the side not used for training). The attention weight calculation formula is as follows: (in 3D feature vectors);

[0169] Fully connected layer: contains two hidden layers (the first layer has 512 neurons and uses the ReLU activation function; the second layer has 256 neurons and uses the LeakyReLU activation function), which perform non-linear transformation on the attention-weighted features to achieve deep fusion of two-dimensional coordinate features and contextual features;

[0170] The 3D coordinate output layer consists of a 128-neuron hidden layer (ReLU activation) and a 48-neuron output layer (Linear activation), outputting the 3D coordinates of 8 keypoints (each keypoint contains x, y, and z dimensions, for a total of 24 dimensions). Model training: Based on the "2D coordinates - 3D coordinates" paired dataset (500 sets, 300 frames per set) prepared in step 1.3, fine-tuning was performed for 100 rounds.

[0171] Hyperparameters: learning rate 1e-4, batch size 16, loss function is mean squared error (MSE), optimizer is Adam;

[0172] Training monitoring: Validation is performed every 10 rounds. Training stops when the validation set MSE drops from the initial 0.01 to 0.0004. At this point, the 3D coordinate prediction error is controlled within ±0.02m, which meets the requirements for action representation.

[0173] ② Establishment of local coordinate system and output of 3D coordinates:

[0174] Establish a local coordinate system with the midpoint of the line connecting the key points of the left and right hips as the origin (since upper limb movements are supported by the upper body, this origin can eliminate the influence of body position movement):

[0175] X-axis: Along the left-right direction of the human body (positive to the right), corresponding to the horizontal movement of the upper limbs;

[0176] Y-axis: along the vertical direction of the human body (upward is positive), corresponding to the vertical movement of the upper limbs lifting;

[0177] Z-axis: Along the front-back direction of the human body (forward is positive), corresponding to the front-back offset of the upper limbs.

[0178] Input the 300 frames of 2D coordinates output from step 2 into the trained model, and it will output the 3D relative coordinates of 8 key points. For example:

[0179] Frame 1 (elbow bent at 90°): Right shoulder coordinates (0.00m, 0.00m, 0.00m), right elbow coordinates (0.12m, 0.35m, 0.21m), right wrist coordinates (0.25m, 0.58m, 0.32m);

[0180] Frame 300 (elbow extended 180°): Right shoulder coordinates (0.00m, 0.00m, 0.00m), right elbow coordinates (0.11m, 0.32m, 0.20m), right wrist coordinates (0.10m, 0.18m, 0.19m). The 3D coordinates are stored as a JSON file in the format "frame number: {keypoint name: [x, y, z], ...}", used for joint angle calculation in step 4.

[0181] Step 4: Calculate the rotation angle of the joint.

[0182] This step calculates the rotation angle of the upper limb core joints using spatial geometric formulas based on three-dimensional relative coordinates. This angle is applicable to universal movement representations across different body proportions and can also be directly used as a robot motor control parameter. Figure 5As shown, the specific execution process is as follows:

[0183] ① Establishing the relationship between joints and key points: Based on the human skeletal structure, identify the two core joints (shoulder and elbow) involved in upper limb movements and their corresponding parent and child key points:

[0184] Right elbow joint: The parent key point is the right shoulder (the fulcrum of the movement), and the child key point is the right wrist (the execution end of the movement). The elbow flexion and extension angles are calculated through the vector relationship of "right shoulder-right elbow-right wrist".

[0185] Right shoulder joint: The parent key point is the left shoulder (reference fulcrum, since the position of the left shoulder is relatively fixed), the child key point is the right shoulder (the joint itself), and the auxiliary key point is the right wrist (to determine the shoulder rotation plane). The shoulder rotation angle is calculated through the vector relationship of "left shoulder - right shoulder - right wrist".

[0186] ② Calculation of key-point correlation vectors:

[0187] Extracting 3D coordinates:

[0188] Right shoulder (parent key point P): ;

[0189] Right elbow (joint J): ;

[0190] Right wrist (sub-key point C): .

[0191] Calculate the vector:

[0192] Vector from parent keypoint to joint: ;

[0193] Vector from keypoint to subkeypoint: .

[0194] ③ Solving for the rotation angle of the joint: The vector is calculated using the dot product formula of spatial vectors. and The included angle is the joint rotation angle θ, and the calculation formula is: .

[0195] Based on human anatomy corrections, the angles of the right elbow and right shoulder joints are calculated over 300 frames, and the result is a CSV file containing "frame number - elbow angle - shoulder angle".

[0196] Step 5: Motion time series construction.

[0197] This step arranges the joint angle data in chronological order along the video timeline, forming a motion time sequence that reflects the temporal changes of the movement, such as... Figure 6 As shown, the specific execution process is as follows:

[0198] 1. Timeline alignment.

[0199] Using the video frame number as the time index (1-300), corresponding to actual time 0-10 seconds (time interval 0.033 seconds / frame), arrange the 300 frames of joint angle data output in step 4 in chronological order:

[0200] Time series of right elbow joint : (300 data points) reflecting the change in angle from bending to straightening;

[0201] Time series of right shoulder joints (300 data points) reflecting the changes in shoulder rotation angle during the lifting process.

[0202] 2. Motion time series integration.

[0203] Will and Combined into a complete motion time series The data is stored as an Excel file, containing four columns: "Time Index (frame), Actual Time (seconds), Right Elbow Angle (°), Right Shoulder Angle (°)". This sequence clearly presents the temporal characteristics of the movement.

[0204] First 50 frames (0-1.65 seconds): The elbow slowly bends and adjusts, with the angle stabilizing at 90°±3°;

[0205] Frames 51-250 (1.65-8.25 seconds): Core lifting phase, elbow angle increases linearly from 90° to 175°, shoulder angle increases from 15° to 38°;

[0206] Frames 251-300 (8.25-10 seconds): The extension stabilization period, with the elbow angle maintained at 178°±2° and the shoulder angle maintained at 39°±1°.

[0207] Step 6: Post-processing of motion time series.

[0208] This step eliminates detection errors and timing jitter by removing dirty data and smoothing motion, improving motion data quality and ensuring stable robot execution. Figure 6 As shown, the specific execution process is as follows:

[0209] 1. Dirty data removal (based on...) in principle).

[0210] For each joint's time series, calculate the mean and standard deviation. Remove excess .

[0211] Outliers in the range:

[0212] Right elbow time series Calculated The abnormal range is ;

[0213] Outlier detected: The right elbow angle in frame 45 is 235.1° (exceeding the upper limit of 220.5°), which is determined to be dirty data (due to occlusion during key point detection). It is removed from T_1.

[0214] Missing value completion: Using linear interpolation, based on data from frame 44 (102.5°) and frame 46 (98.7°), the angle in frame 45 was completed. This ensures the continuity of the time series.

[0215] 2. Motion smoothing (sliding window filtering).

[0216] Considering the upper limb lifting motion as a medium-speed smoothed movement, a sliding window size of N=5 (an odd number to balance smoothing effect and preservation of movement details) is selected. The time series after removing dirty data is then processed as follows:

[0217] Filtering formula: For the j-th data point in the time series Take N consecutive data points before and after the first (if there are fewer than N points at the beginning and end, fill in the boundary data repeatedly), and calculate the mean as the smoothed data point. The formula is: ;

[0218] Example: The original angle of the right elbow in frame 10 is 112.3°, so data from frames 8-12 is used. The smoothed angle is ;

[0219] Verification: After smoothing, the angle jitter amplitude of T_1 decreased from ±5.3° to ±0.4°, and T_2 decreased from ±3.1° to ±0.3°, which is consistent with the physical characteristics of robot motion execution (avoiding stuttering caused by frequent motor start-stop).

[0220] The post-processed motion time series is stored as "smoothed_time_series.csv" and used for reward function fitting in step 7.

[0221] Step 7: Reward function fitting.

[0222] This step employs a multinomial regression algorithm (due to the moderate complexity of the upper limb lifting motion, the scatter data is approximately linear) to fit the post-processed motion time series scatter points into a continuous reward function, providing accurate reward feedback for robot reinforcement learning, such as... Figure 6 As shown, the specific execution process is as follows:

[0223] 1. Construction of fitted data samples.

[0224] Using "robot joint angle deviation value" as the input feature and "action similarity reward value" as the output label, 1000 sets of fitted samples are constructed:

[0225] Input features (deviation values) ): ,in Let be the angle of the i-th joint of the robot at time t (simulated value, range 0°-180°). The angle of the human joint after post-processing in step 6 (e.g., the angle of the human right elbow is 95.2° at t=1 second).

[0226] Output label (reward value r): Based on the deviation value, the smaller the deviation, the greater the reward. ( (Determined through cross-validation) as the initial label, for example:

[0227] (Exact match): (Maximum reward);

[0228] (Small deviation): ;

[0229] (Big deviation): (Close to 0);

[0230] (Serious deviation): (Negative punishment)

[0231] Ultimately, 1000 groups were formed. Scattered samples, of which 800 sets are used for training and 200 sets are used for validation.

[0232] 2. Polynomial regression fitting.

[0233] Assume the reward function is a 5th-order polynomial (by trying polynomials of orders 3-7, the 5th-order polynomial has the smallest validation set mean square error, at 0.32): The coefficient vector is solved using the least squares method. ;

[0234] Construct the design matrix X: an 800×6 dimensional matrix, with the k-th row... ;

[0235] Construct a label vector y: an 800×1 dimensional matrix, to store the reward values ​​r of 800 training samples;

[0236] Solving the normal system of equations The calculation yields: ;

[0237] The fitted reward function is stored as a Python function file "reward_function.py" for use in the reinforcement learning training in step 8.

[0238] Step 8: Robot reinforcement learning training.

[0239] This step, based on the PPO algorithm (adapted to the real-time requirements of robots and with fast training convergence), combined with the reward function from step 7, trains the upper limb lifting motion model of the rehabilitation robot, such as... Figure 7 As shown, the specific execution process is as follows:

[0240] 1. Definition of state space and action space.

[0241] State space S: defined as the vector of angular deviations between the robot's two core joints (elbow and shoulder) and the target human motion, i.e. ,in For elbow deviation, For shoulder deviation, the state space dimension is 2;

[0242] Action space A: defined as the angle adjustment vector of two joints of the robot, i.e. ,in (Elbow adjustment amount) (Because the maximum single adjustment of the elbow motor is 3°). (Shoulder adjustment amount) (Shoulder motor limitation), the motion space dimension is 2.

[0243] 2. PPO algorithm initialization.

[0244] Network Structure: The policy network and value network share the first half of the feature extraction layer (2 fully connected layers, 512→256 neurons). The output layer of the policy network has 2 neurons (corresponding to the adjustment of 2 joints), and the output layer of the value network has 1 neuron (corresponding to the state value). );

[0245] Hyperparameter: Clip coefficient (Limit the policy update range to avoid training instability), learning rate Batch size B=128, trajectory length per round T=2048, number of iterations per round K=5;

[0246] Agent initialization: The robot's initial state is with all joints at zero (elbow 0°, shoulder 0°), initial deviation value. (The elbow needs to be adjusted from 0° to 92.3°). (The shoulder angle needs to be adjusted from 0° to 15.2°).

[0247] 3. The training loop is executed.

[0248] A total of 1000 training rounds were executed, with each round following the following procedure:

[0249] Trajectory acquisition: The agent in its current state Below, the action probability distribution is output through the policy network, and actions are sampled. And execute (e.g., step 1 of round 1, action) );

[0250] State transition and reward calculation: Robot performs actions Then, transition to the new state. (For example, if the elbow angle changes from 0° to 2.5°, the deviation) The reward function in step 7 is called to calculate the immediate reward. (like (Due to the large initial deviation, the reward is close to 0).

[0251] Track storage: Recording track data This continues until 2048 trajectory data points are collected;

[0252] Dominance function calculation: The dominance function is calculated using the time-difference (TD) method. ( (discount factor), and the dominance function is standardized (mean is 0, standard deviation is 1);

[0253] Strategy and Value Network Update:

[0254] Policy network update: via Clip loss function (where the probability ratio is used), minimize the loss and update the policy parameters;

[0255] Value network update: via mean squared error loss ( (For the target value), update the value network parameters;

[0256] Convergence determination: The training is determined to be converged when the average reward of the policy network increases from the initial -2.3 to 8.5 in 100 consecutive rounds and the fluctuation range is less than 3%, and the KL divergence of the policy network (the difference from the initial policy) is less than 0.01.

[0257] 4. Output of the target action model.

[0258] After training convergence, the policy network parameters are saved to form an upper limb lifting motion model adapted to the rehabilitation robot. This model can output the optimal motion adjustment amount based on the robot's real-time state, ensuring that the robot's movements quickly converge to the target human movements.

[0259] The implementation of the various embodiments of the present invention is based on programmed processing by a device with processor functionality. Therefore, in practical engineering, the technical solutions and functions of the various embodiments of the present invention are encapsulated into various modules. Based on this reality, and building upon the above embodiments, the embodiments of the present invention provide a robot motion training system based on human motion videos. This system is used to execute a robot motion training method based on human motion videos from the above method embodiments.

[0260] The system includes: a video preprocessing module for acquiring and preprocessing human motion videos; a keypoint detection module for detecting keypoints in the preprocessed human motion video frames using keypoint capture technology and outputting two-dimensional image coordinates of keypoints in the video; a three-dimensional coordinate mapping module for inputting the two-dimensional image coordinates of keypoints into a deep learning coordinate mapping model and outputting the three-dimensional relative coordinates of keypoints, where the three-dimensional relative coordinates only reflect the relative positional relationship between keypoints; a joint angle calculation module for obtaining the rotation angle information of each joint point through spatial geometric calculation based on the output three-dimensional relative coordinates of each keypoint; a time series construction and post-processing module for arranging the obtained rotation angle information of each joint point in the order of the video timeline to construct a motion time series and perform post-processing; a reward function fitting module for fitting the scattered data corresponding to the post-processed motion time series into a reward function using a function fitting algorithm, where the reward function is used to calculate the reward value during the robot's reinforcement learning training process; and a reinforcement learning training module for training the robot based on the obtained reward function to obtain a target action model adapted to the robot's execution.

[0261] This invention provides a robot motion training system based on human motion videos. Addressing the problems in existing robot motion training and scheduling technologies, such as insufficient accuracy in motion feature extraction, poor universality of motion representation, low quality of motion time series, unreasonable reward function design, and poor convenience of motion model scheduling, this system employs the aforementioned modules to accurately extract motion features from ordinary human motion videos, achieve universal motion representation, improve motion data quality, optimize the design of reinforcement learning reward functions, and realize standardized scheduling of motion models for robot motion training.

[0262] It should be noted that the system embodiments provided by the present invention are used not only to implement the methods in the above method embodiments, but also to implement the methods in other method embodiments provided by the present invention. The only difference is that corresponding functional modules are set. The principle is basically the same as that of the above system embodiments provided by the present invention. As long as those skilled in the art can improve the modules in the above system embodiments by referring to the specific technical solutions in other method embodiments and combining technical features to obtain corresponding technical means and technical solutions composed of these technical means, on the basis of the above system embodiments, and on the premise of ensuring the practicality of the technical solutions, they can obtain corresponding system-like embodiments for implementing the methods in other method-like embodiments.

[0263] In summary, this invention discloses a robot motion training method and system based on human motion videos. First, the human motion video is preprocessed to extract and enhance video frames. Then, the two-dimensional coordinates of key motion points are detected and mapped to three-dimensional relative coordinates to calculate joint rotation angles. Next, a motion time series is constructed and optimized, and a reward function is fitted. Finally, a target motion model is obtained through training using a reinforcement learning algorithm. The system core relies on deep learning networks including ResNet backbone layers and the DDPG / PPO reinforcement learning framework. This invention addresses existing pain points based on a complete technical process, achieving precise robot motion imitation through rigorous steps. The solution is reproducible, scalable, and applicable to robot motion training needs in various fields such as industrial collaboration, service, and rehabilitation.

[0264] The terms “comprising” and “having”, and any variations thereof, in the specification, claims, and accompanying drawings of this invention are intended to cover a non-exclusive inclusion, such as a process, method, system, product, or apparatus that includes a series of steps or units, not necessarily limited to those explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0265] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. A method for training robot motion based on human motion videos, characterized in that, include: Acquire and preprocess videos of human movement; Key point capture technology is used to detect preprocessed human motion video frames and output two-dimensional image coordinates of each key point of the human in the video. The two-dimensional image coordinates of each key point are input into a deep learning coordinate mapping model, and the three-dimensional spatial relative coordinates of each key point are output. The three-dimensional spatial relative coordinates only reflect the relative positional relationship between the key points of the human body. Based on the three-dimensional spatial relative coordinates of each key point, the rotation angle information of each joint point is obtained through spatial geometric calculation. The calculation of the rotation angle information of the joint points is based on the spatial vector angle formula, including: determining the parent and child key points corresponding to each joint point according to the three-dimensional spatial relative coordinates of the key points, and establishing the joint point-key point association relationship; calculating the vector from the parent key point to the joint point corresponding to each joint point. And the vector from the keypoint to the subkeypoint The vector is calculated using the dot product formula of spatial vectors. and The included angle, i.e. the rotation angle of the joint point; The obtained rotation angle information of each joint point is arranged in the order of the video timeline to construct a motion time series and then post-processed. A function fitting algorithm is used to fit the scatter data corresponding to the post-processed motion time series into a reward function. The input of the reward function is the joint rotation angle deviation between the robot's current action state and the human target action, and the output is the corresponding reward value. The reward function is used to calculate the reward value in the robot's reinforcement learning training process. Based on the obtained reward function, the robot is trained using reinforcement learning to obtain a target action model that is adapted to the robot's execution.

2. The robot motion training method based on human motion videos according to claim 1, characterized in that, The preprocessing includes video frame extraction, human target detection and segmentation, and image enhancement.

3. The robot motion training method based on human motion videos according to claim 1, characterized in that, The key point capture technology employs a deep learning network based on skeletal key point detection, which includes a feature extraction backbone layer, a key point prediction branch layer, and a coordinate regression layer. The feature extraction backbone layer adopts a residual neural network structure to extract deep semantic features of human targets in video frames; The key point prediction branch layer adopts a structure combining convolutional and deconvolutional layers to upsample and refine deep semantic features and output a key point heatmap. The coordinate regression layer is based on the key point heatmap. It uses a peak detection algorithm to locate the center position of each key point and then calculates the two-dimensional image coordinates of the key points. The key points include at least the core motion-related key points of the head vertex, neck, shoulder, elbow, wrist, hip, knee and ankle.

4. The robot motion training method based on human motion videos according to claim 3, characterized in that, The deep learning coordinate mapping model is an end-to-end three-dimensional coordinate regression network, including an input layer, a feature fusion layer, and a three-dimensional coordinate output layer. The input layer receives the two-dimensional image coordinates of key points and the corresponding video frame context features, which are extracted from the video frames by a convolutional neural network. The feature fusion layer adopts a structure that combines attention mechanism and fully connected layer to fuse two-dimensional image coordinate features and video frame context features, highlighting key feature information for three-dimensional coordinate mapping; The three-dimensional coordinate output layer is implemented through a multilayer perceptron, which outputs the three-dimensional spatial relative coordinates of each key point. The three-dimensional spatial relative coordinates establish a local coordinate system with the key point of the human hip as the origin. The X-axis is along the left and right direction of the human body, the Y-axis is along the up and down direction of the human body, and the Z-axis is along the front and back direction of the human body.

5. The robot motion training method based on human motion videos according to claim 1, characterized in that, The function fitting algorithm adopts either Gaussian process regression or polynomial regression.

6. The robot motion training method based on human motion videos according to claim 5, characterized in that, The method further includes: When using the Gaussian process regression algorithm, the regression model parameters are determined by maximizing the posterior probability estimate, and the motion time series scatter data is used as training samples to fit a continuous reward function curve. When using the polynomial regression algorithm, the polynomial coefficients are solved by the least squares method, and the degree of the polynomial is determined according to the motion complexity of the motion time series.

7. The robot motion training method based on human motion videos according to claim 1, characterized in that, The post-processing includes dirty data removal and motion smoothing. Dirty data removal is achieved through an outlier detection algorithm, and motion smoothing is completed through a time-series data filtering algorithm.

8. The robot motion training method based on human motion videos according to claim 1, characterized in that, The reinforcement learning training employs a deep deterministic policy gradient algorithm or a proximal policy optimization algorithm, including: Construct the robot's motion space and state space. The motion space is the feasible domain of the rotation angle of each joint, and the state space is the deviation between the robot's current joint rotation angle and the target motion joint rotation angle. The obtained reward function is integrated into the reward mechanism of reinforcement learning. When the deviation between the robot's action state and the target action state decreases, the reward function outputs a positive reward value, and when the deviation increases, it outputs a negative reward value. Initialize the parameters of the reinforcement learning agent and set the hyperparameters; During training, the agent interacts with the environment, selects actions based on the current state, obtains reward values ​​through a reward function, and updates the agent's policy network parameters using the gradient descent algorithm until training converges.

9. A robot motion training system based on human motion videos, characterized in that, include: The video preprocessing module is used to acquire and preprocess videos of human movement. The key point detection module is used to detect preprocessed human motion video frames using key point capture technology and output the two-dimensional image coordinates of each key point of the human in the video. The three-dimensional coordinate mapping module is used to input the two-dimensional image coordinates of each key point into the deep learning coordinate mapping model and output the three-dimensional spatial relative coordinates of each key point. The three-dimensional spatial relative coordinates only reflect the relative positional relationship between the key points of the human body. The joint angle calculation module is used to obtain the rotation angle information of each joint point through spatial geometric calculation based on the three-dimensional spatial relative coordinates of each key point output. The calculation of the rotation angle information of the joint points is based on the spatial vector angle formula, including: determining the parent and child key points corresponding to each joint point according to the three-dimensional spatial relative coordinates of the key points, and establishing the joint point-key point association relationship; calculating the vector from the parent key point to the joint point corresponding to each joint point. And the vector from the keypoint to the subkeypoint The vector is calculated using the dot product formula of spatial vectors. and The included angle, i.e. the rotation angle of the joint point; The time series construction and post-processing module is used to arrange the obtained rotation angle information of each joint point in the order of the video timeline, construct the motion time series and perform post-processing. The reward function fitting module is used to fit the scatter data corresponding to the post-processed motion time series into a reward function using a function fitting algorithm. The input of the reward function is the joint rotation angle deviation value between the robot's current action state and the human target action, and the output is the corresponding reward value. The reward function is used to calculate the reward value during the robot's reinforcement learning training process. The reinforcement learning training module is used to train the robot through reinforcement learning based on the obtained reward function, so as to obtain a target action model that is suitable for the robot to perform.

Images