A robot skill learning method and system using multi-source video data

Abstract

The present invention provides a multi-source video data robot skill learning method and system; through an example video collection module, example videos related to the skill are automatically collected and data expansion is performed according to a text description of the sports skill, so as to obtain sports example video data; a virtual robot and a virtual camera are constructed and instantiated, a robot control strategy and a camera movement strategy are coordinated and an intelligent body is combined to generate and record robot motion video recording data; a video intelligent scoring model is constructed through a sports skill video scoring module to generate scoring results for the robot motion video recording data; through an intelligent body learning module, a reward feedback of a neural network scoring model is set to coordinately optimize the robot control strategy and the camera movement strategy, and update them to the intelligent body.

Description

Technical Field

[0001] The present invention relates to the field of machine learning intelligent iterative data processing information transmission control technology, and more specifically, to a multi-source video data robot skill learning method and system. Background Art

[0002] The skill learning of virtual robots aims to enable them to have a variety of sports skill keywords (such as walking, running, grabbing objects, etc.). These sports skill keywords can be used to generate robot character animations that conform to human movement patterns and physical laws. Since the robot's movement process often requires the joint control of multiple joints, and the collision interaction with the environment during the movement is usually non-differentiable, existing methods usually use reinforcement learning technology to learn the robot's control strategy. The training signal of the reinforcement learning method comes from the reward feedback obtained after the robot performs a certain action, and the robot's control strategy is gradually adjusted according to the feedback to obtain a higher expected reward. The imitation learning method provides a simple and effective reward calculation method, that is, the reward is calculated based on the similarity between the robot's action sequence and the example action sequence (usually from human examples): the higher the similarity, the greater the reward; conversely, the lower the similarity, the smaller the reward;

[0003] There are two commonly used methods to calculate the similarity between the robot's action sequence and the example action sequence: one is based on tracking error, that is, calculating the average L2 distance between each joint of the robot and the joints of the example individual at the corresponding moment; the other is based on the distance between distributions, that is, measuring the distance between the state transition distribution generated when the robot moves and the state transition distribution generated by the example individual, which can be quantified and estimated by the discriminator in the adversarial learning method. The premise for calculating these two rewards is to obtain the three-dimensional motion trajectory of each joint of the example individual, which often needs to be restored by a motion capture device or motion reconstruction algorithm. Existing skill learning methods based on video data usually include two steps: first, reconstruct the motion posture sequence of the example individual from the example video, and then calculate the reward signal for the robot based on tracking error or distance between distributions. Since both motion capture and motion reconstruction methods have high requirements for training video acquisition, most of the existing video skill learning methods are only suitable for videos recorded with fixed perspectives and small character movements. Therefore, improving the existing video skill learning methods to better utilize massive Internet data is an important research challenge; existing video skill learning methods usually rely on three-dimensional pose sequences for training, that is, calculating reward signals in three-dimensional space to assist robot learning; however, three-dimensional pose sequences usually require the use of complex motion capture equipment, or motion reconstruction algorithms to restore; both methods have high requirements on the acquisition process of training videos; this limitation makes it difficult for existing algorithms to effectively utilize the massive sports skill demonstration videos on the Internet, thereby limiting the range of skills that robots can learn, and other problems remain to be solved; therefore, it is necessary to propose a multi-source video data robot skill system and method to at least partially solve the problems existing in the prior art. Summary of the invention

[0004] A series of simplified concepts are introduced in the content of the invention, which will be further described in detail in the specific implementation method section; the content of the invention of the present invention does not mean to attempt to limit the key features and essential technical features of the technical solution claimed for protection, nor does it mean to attempt to determine the scope of protection of the technical solution claimed for protection.

[0005] To at least partially solve the above problems, the present invention provides a method for robot skill learning using multi-source video data, comprising:

[0006] S100, automatically collecting sample videos related to the skill according to the text description of the sport skill through a sample video collection module and performing data expansion to obtain sport sample video data;

[0007] S200, constructing and instantiating a virtual robot and a virtual camera, coordinating the robot control strategy and the camera movement strategy and combining the intelligent body, generating and recording robot motion video recording data;

[0008] S300, through the sports skill video scoring module, builds a video intelligent scoring model to generate scoring results for the robot sports video recording data;

[0009] S400, through the agent learning module, sets the reward feedback of the neural network scoring model to collaboratively optimize the robot control strategy and camera movement strategy, and updates them to the agent.

[0010] Preferably, S100 includes:

[0011] S101, setting a sports skill text description; parsing and extracting sports skill keywords from the sports skill text description by using a keyword extraction algorithm in combination with a large language model;

[0012] S102, performing tag content aggregation on sports skill keywords to obtain sports skill tags;

[0013] S103, according to the sports skill keywords and sports skill tags, collect sports example videos and perform data expansion to obtain sports example video data and sports video expansion data.

[0014] Preferably, S200 includes:

[0015] S201, using the simulation experiment platform, based on the robot parameter information, performing robot virtual modeling, and constructing a virtual robot composed of a plurality of rigid bodies and a virtual structure;

[0016] S202, performing camera modeling according to the camera parameter information, constructing a pinhole model camera with a collision-free property, and obtaining a virtual camera;

[0017] S203, instantiate the virtual robot and the virtual camera, coordinate the robot control strategy with the camera movement strategy, and combine the instantiated virtual robot and virtual camera into an intelligent entity, perform virtual robot control and motion video recording, generate and record the robot motion video captured by the camera in the simulation environment, and obtain the robot motion video recording data; transmit the robot motion video recording data to the sports skill video scoring module.

[0018] Preferably, S300 includes:

[0019] S301, extracting and storing features of robot motion video recording data, robot motion example video data, and motion video expansion data;

[0020] S302, constructing a neural network scoring model, comparing the feature similarity between the robot motion video recording data and the robot motion example video data and the motion video expansion data, and generating a scoring result for the motion video recording data.

[0021] Preferably, S400 includes:

[0022] S401, the agent learning module coordinately optimizes the robot control strategy and the camera movement strategy based on the feedback of the scoring result of the motion video recording data of the neural network scoring model through the reward feedback of the neural network scoring model, and obtains the optimized robot control strategy and the optimized camera movement strategy;

[0023] S402, iteratively optimize the agent strategy according to the optimized robot control strategy and the optimized camera movement strategy, and update it to the virtual robot and virtual camera combined agent.

[0024] The present invention provides a multi-source video data robot skill system, comprising:

[0025] The video collection data expansion subsystem automatically collects sample videos related to the skill and performs data expansion based on the text description of the sport skill through the sample video collection module to obtain the sport sample video data;

[0026] Virtually construct the control strategy subsystem, build and instantiate the virtual robot and virtual camera, coordinate the robot control strategy and camera movement strategy and combine the intelligent body, generate and record the robot motion video recording data;

[0027] The sports skill intelligent scoring subsystem builds a video intelligent scoring model through the sports skill video scoring module to generate scoring results for the robot's sports video recording data;

[0028] The strategy optimization agent update subsystem sets the reward feedback of the neural network scoring model through the agent learning module to collaboratively optimize the robot control strategy and camera movement strategy, and updates them to the agent.

[0029] Preferably, the video collection data expansion subsystem includes:

[0030] The keyword extraction and parsing unit sets a sports skill text description; the sports skill keywords in the sports skill text description are parsed and extracted from the sports skill text description by using a keyword extraction algorithm combined with a large language model;

[0031] A tag content aggregation unit aggregates tag content of sports skill keywords to obtain sports skill tags;

[0032] The video tag expansion unit collects sports example videos and performs data expansion based on sports skill keywords and sports skill tags to obtain sports example video data and sports video expansion data.

[0033] Preferably, the virtual construction control strategy subsystem includes:

[0034] The robot structure modeling unit uses the simulation experiment platform to perform robot virtual modeling according to the robot parameter information, and constructs a virtual robot composed of multiple rigid bodies.

[0035] A virtual camera modeling unit performs camera modeling according to camera parameter information, constructs a pinhole model camera with a collision-free property, and obtains a virtual camera;

[0036] Instantiate the intelligent agent combination unit, instantiate the virtual robot and the virtual camera, coordinate the robot control strategy with the camera movement strategy, and combine the instantiated virtual robot and the virtual camera into an intelligent agent to control the virtual robot and record motion video, generate and record the robot motion video captured by the camera in the simulation environment, and obtain the robot motion video recording data; transmit the robot motion video recording data to the sports skill video scoring module.

[0037] Preferably, the sports skill intelligent scoring subsystem includes:

[0038] A feature extraction and storage unit is used to extract and store features of robot motion video recording data, robot motion example video data, and motion video expansion data;

[0039] The scoring model feature scoring unit constructs a neural network scoring model, compares the feature similarity between the robot motion video recording data and the robot motion example video data and the motion video expansion data, and generates the motion video recording data scoring result.

[0040] Preferably, the strategy optimization agent update subsystem includes:

[0041] The intelligent agent learning optimization unit, the intelligent agent learning module is based on the feedback of the scoring results of the motion video recording data of the neural network scoring model, and the reward feedback of the neural network scoring model is used to coordinately optimize the robot control strategy and the camera movement strategy to obtain the optimized robot control strategy and the optimized camera movement strategy;

[0042] The intelligent agent strategy iteration and updating unit iteratively optimizes the intelligent agent strategy according to the optimized robot control strategy and the optimized camera movement strategy, and updates it to the virtual robot and virtual camera combined intelligent agent.

[0043] Compared with the prior art, the present invention has at least the following beneficial effects:

[0044] The present invention discloses a method and system for learning robot skills using multi-source video data. The method comprises the following steps: automatically collecting example videos related to the skill and performing data expansion according to a text description of a sport skill through an example video collection module, thereby obtaining example sport video data; constructing and instantiating a virtual robot and a virtual camera, coordinating a robot control strategy with a camera movement strategy, and combining an intelligent agent to generate and record robot sport video recording data; constructing a video intelligent scoring model through a sport skill video scoring module, thereby generating scoring results for robot sport video recording data; and setting a reward feedback of a neural network scoring model through an intelligent agent learning module to collaboratively optimize the robot control strategy and the camera movement strategy, and updating the intelligent agent to the intelligent agent. Energy body; This patent proposes a skill learning method based on multi-source video data, aiming to enable a virtual robot to learn a unified motion control strategy from multi-source video examples; this method not only trains the robot's control strategy, but also trains the camera movement strategy to record the robot, and scores the action through the recording results to assist its learning; this method does not need to restore the three-dimensional motion trajectory in the example video for training, which significantly reduces the complexity of data acquisition and preprocessing, thereby effectively utilizing multi-source video data for learning; this method effectively avoids the data heterogeneity problem caused by different sources of training data, so that the virtual robot can learn unified motion skill keywords from massive example videos from different sources. The method simplifies the complex process of traditional methods in the training data collection and preprocessing stages, and improves the processing efficiency of training data; multi-source video data type; the method models the skill learning problem based on multi-source video data as a multi-agent reinforcement learning problem. While training the robot motion control strategy, it also trains the camera movement strategy to assist the robot in completing skill learning; the camera movement strategy trained by this method does not require human control, and can adjust its own recording trajectory according to the robot's motion trajectory, and naturally display the skill learning results of the virtual robot; a method for learning unified motion skill keywords from multi-source video data; a method for jointly training robot motion control and camera movement strategies; a motion skill keyword learning method based on multi-source video data is implemented. The method can use multi-source motion skill keyword videos collected from the Internet to supervise the virtual robot to obtain new motion skill keywords through imitation learning; since the method does not require the collection of training videos in a specific way, it can use massive network data to learn diverse skills; in addition, the method can also generate a natural camera movement strategy while training the robot to learn motion skill keywords. The strategy can record motion videos for the robot to intuitively display its learning results.

[0045] The multi-source video data robot skill learning method and system described in the present invention, other advantages, objectives and features of the present invention will be reflected in part through the following description, and in part will also be understood by technical personnel in the field through research and practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0047] Figure 1 This is a diagram of an embodiment of a multi-source video data robot skill learning system described in the present invention.

[0048] Figure 2 This is a diagram showing an embodiment of a multi-source video data robot skill learning method described in the present invention.

[0049] Figure 3 This is a diagram of an example implementation example of a multi-source video data robot skill learning method and system simulation according to the present invention. DETAILED DESCRIPTION

[0050] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments so that those skilled in the art can implement it according to the instructions; as shown in the figure, the present invention provides a multi-source video data robot skill learning method, comprising:

[0051] S100, automatically collecting sample videos related to the skill according to the text description of the sport skill through a sample video collection module and performing data expansion to obtain sport sample video data;

[0052] S200, constructing and instantiating a virtual robot and a virtual camera, coordinating the robot control strategy and the camera movement strategy and combining the intelligent body, generating and recording robot motion video recording data;

[0053] S300, through the sports skill video scoring module, builds a video intelligent scoring model to generate scoring results for the robot sports video recording data;

[0054] S400, through the agent learning module, sets the reward feedback of the neural network scoring model to collaboratively optimize the robot control strategy and camera movement strategy, and updates them to the agent.

[0055] The principle and effect of the above technical solution are as follows: The present invention provides a multi-source video data robot skill learning method, including: automatically collecting example videos related to the skill and performing data expansion according to the text description of the sports skill through the example video collection module to obtain sports example video data; constructing and instantiating a virtual robot and a virtual camera, coordinating the robot control strategy with the camera movement strategy and combining the intelligent body, generating and recording the robot motion video recording data; constructing a video intelligent scoring model through the sports skill video scoring module to generate the scoring results for the robot motion video recording data; setting the reward feedback of the neural network scoring model through the intelligent body learning module to coordinately optimize the robot control strategy and the camera movement strategy and update it to the intelligent agent; this patent proposes a skill learning method based on multi-source video data, which aims to enable the virtual robot to learn a unified motion control strategy from multi-source video examples; this method not only trains the robot's control strategy, but also trains the camera movement strategy to record the robot, and scores the actions through the recording results to assist its learning; this method does not need to restore the three-dimensional motion trajectory in the example video for training, which significantly reduces the complexity of data acquisition and preprocessing, thereby effectively utilizing multi-source video data for learning; this method effectively avoids the data heterogeneity problem caused by different sources of training data, allowing the virtual robot to learn unified motion skill keywords from massive example videos from different sources. This method simplifies the complex process of traditional methods in the training data collection and preprocessing stages, and improves the processing efficiency of training data; multi-source video data type; this method models the skill learning problem based on multi-source video data as a multi-agent reinforcement learning problem. While training the robot motion control strategy, it also trains the camera movement strategy to assist the robot in completing skill learning; the camera movement strategy trained by this method does not require human control, and can adjust its own recording trajectory according to the robot's motion trajectory, naturally showing the skill learning results of the virtual robot; a method for learning unified motion skill keywords from multi-source video data; a method for jointly training robot motion control and camera movement strategies; a motion skill keyword learning method based on multi-source video data is implemented. This method can use multi-source motion skill keyword videos collected from the Internet to supervise the virtual robot to obtain new motion skill keywords through imitation learning; since this method does not require the collection of training videos in a specific way, it can use massive network data to learn diverse skills; in addition, this method can also generate a natural camera movement strategy while training the robot to learn motion skill keywords. This strategy can record motion videos for the robot to intuitively show its learning results; such as Figure 1As shown, the example video collection module collects motion example videos through big data tag content capture, and analyzes and determines the motion keyword information of the motion example videos; the motion keyword information includes: motion description keywords and motion action feature keywords; the motion description keywords perform video text description analysis through the text description content of the motion example videos and the subtitle text description content to obtain the video text description analysis results; and compare the video text description analysis results with the motion action feature keywords to determine whether the video text description and the motion action features are consistent; if the video text description and the motion action features are consistent, the motion example video data is transmitted to the skill video scoring module; the skill video scoring module receives the 3D simulation data in parallel. The skill video recording module in the environment system records the robot motion video recording data; the virtual robot and the virtual camera are combined to construct a combined intelligent body, the combined intelligent body records the virtual robot video, and obtains the robot motion video recording data; the robot motion video recording data is transmitted to the skill video scoring module; the skill video scoring module scores the motion example video data and the robot motion video recording data, and obtains the robot motion video recording data scoring information; the robot motion video recording data scoring information is transmitted to the intelligent body learning module for intelligent learning training; the intelligent learning training information is continuously transmitted to the 3D simulation environment, and the 3D simulation environment and the combined intelligent body are coordinated and optimized according to the intelligent learning training information.

[0056] In one embodiment, S100 includes:

[0057] S101, setting a sports skill text description; parsing and extracting sports skill keywords from the sports skill text description by using a keyword extraction algorithm in combination with a large language model;

[0058] S102, performing tag content aggregation on sports skill keywords to obtain sports skill tags;

[0059] S103, according to the sports skill keywords and sports skill tags, collect sports example videos and perform data expansion to obtain sports example video data and sports video expansion data.

[0060] The principle and effect of the above technical solution are: setting a text description of sports skills; parsing and extracting sports skill keywords in the sports skill text description through a keyword extraction algorithm combined with a large language model; performing tag content aggregation on sports skill keywords to obtain sports skill tags; collecting sports example videos and performing data expansion according to sports skill keywords and sports skill tags to obtain sports example video data and sports video expansion data; collecting sports example videos and performing data expansion according to sports skill keywords and sports skill tags to obtain sports example video data and sports video expansion data includes: collecting example videos through data crawling according to sports skill keywords and sports skill tags to obtain collected sports example videos; and matching sports skill tags with reference sports tags in existing sports data sets, taking out sports videos with a matching degree higher than the set reference matching degree for data expansion, and obtaining sports video expansion data; transmitting the sports example video data and sports video expansion data to a skill video scoring module; such as Figure 2 As shown in the figure, by analyzing the text of the skill description, sample video data that meets the description is collected; a simulation environment is constructed and the robot and camera are instantiated, and the camera movement and control strategies are initialized / updated; the robot's motion video is recorded according to the camera movement strategy, and a scoring model is trained in combination with the sample video; the agent strategy is learned and optimized based on the feedback of the scoring model, and updated to the simulation environment.

[0061] In one embodiment, S200 includes:

[0062] S201, using the simulation experiment platform, based on the robot parameter information, performing robot virtual modeling, and constructing a virtual robot composed of a plurality of rigid bodies and a virtual structure;

[0063] S202, performing camera modeling according to the camera parameter information, constructing a pinhole model camera with a collision-free property, and obtaining a virtual camera;

[0064] S203, instantiate the virtual robot and the virtual camera, coordinate the robot control strategy with the camera movement strategy, and combine the instantiated virtual robot and virtual camera into an intelligent entity, perform virtual robot control and motion video recording, generate and record the robot motion video captured by the camera in the simulation environment, and obtain the robot motion video recording data; transmit the robot motion video recording data to the sports skill video scoring module.

[0065] The principle and effect of the above technical solution are as follows: using the simulation experiment platform, according to the robot parameter information, the robot virtual modeling is performed, and a virtual robot with a virtual structure composed of multiple rigid bodies is constructed; according to the camera parameter information, the camera modeling is performed, a pinhole model camera with collision-free properties is constructed, and a virtual camera is obtained; the virtual robot and the virtual camera are instantiated, the robot control strategy and the camera movement strategy are coordinated, and the instantiated virtual robot and the virtual camera are combined into an intelligent body to control the virtual robot and record motion video, generate and record the robot motion video captured by the camera in the simulation environment, and obtain the robot motion video recording data; the robot motion video recording data is transmitted to the sports skill video scoring module; using the simulation experiment platform, according to the robot parameter information, the robot virtual modeling is performed, and a virtual robot with a virtual structure composed of multiple rigid bodies is constructed. 211 includes: using the simulation experiment platform, according to the robot parameter information, a virtual structure composed of multiple rigid bodies is constructed, and the various parts of the virtual structure are connected by virtual joints; the knee joint and the elbow joint are connected by 1 The robot character includes 13 joints and 34 degrees of freedom. The robot entity virtual associated parameters are set. The robot entity virtual associated parameters include: the total mass of the robot is 45 kg; the height of the robot is 1.62 meters; the dimension of the robot state space is 197 dimensions; the dimension of the robot action space is 36 dimensions; the robot control strategy network model is constructed. The robot control strategy network model is constructed by taking the robot state information as input and outputting the torque value of each joint to control the robot state information. The robot state information includes the spatial position and movement speed of each joint of the robot. The control strategy is modeled by a fully connected neural network with input and output dimensions of 197 and 36 respectively to obtain the robot control strategy network model. The training scene is a randomly initialized ground, and the robot performs different actions on the ground. The ground is set with multiple grid types and multiple physical parameters during initialization to improve the robustness of the robot control strategy. The multiple grid types include: flat grid and rugged grid; the multiple physical parameters include: friction coefficient and elastic recovery coefficient.

[0066] According to the camera parameter information, camera modeling is performed to construct a pinhole model camera with collision-free properties, and obtaining a virtual camera 212 includes: randomly sampling and generating the camera's internal parameter parameters within a certain range to cover the camera's internal parameter range used when collecting multi-source video data; constructing a camera movement strategy network model; the input of the camera movement strategy network model includes the world poses of the camera and the robot in the past several frames, the relative poses of the camera and the robot, and the pixel coordinates of the robot's joints projected in the picture; the output of the camera movement strategy network model is the camera's next movement instruction, and the next movement instruction is decomposed into a rotation instruction and a displacement instruction; the virtual The robot and the virtual camera are instantiated, the robot control strategy and the camera movement strategy are coordinated, and the instantiated virtual robot and the virtual camera are combined into an intelligent entity to control the virtual robot and record motion videos, including: the robot and the camera are instantiated for the first time, and the parameters of the robot control strategy network model and the camera movement strategy network model are randomly initialized; in the subsequent learning process, the strategy networks of the robot control strategy network model and the camera movement strategy network model are gradually optimized and updated; the robot control strategy network model enables the robot to accurately reproduce the action skills in the example video, and the camera movement strategy network model records the motion video from an angle similar to the example video.

[0067] In one embodiment, S300 includes:

[0068] S301, extracting and storing features of robot motion video recording data, robot motion example video data, and motion video expansion data;

[0069] S302, constructing a neural network scoring model, comparing the feature similarity between the robot motion video recording data and the robot motion example video data and the motion video expansion data, and generating a scoring result for the motion video recording data.

[0070] The principle and effect of the above technical solution are: extracting and storing features of robot motion video recording data, robot motion example video data and motion video expansion data; building a neural network scoring model, comparing the feature similarity of robot motion video recording data with robot motion example video data and motion video expansion data, and generating a scoring result for the motion video recording data;

[0071] Construct a neural network scoring model, and compare the feature similarity of the robot motion video recording data with the robot motion example video data and the motion video expansion data, including: the input of the neural network scoring model is the projection feature of the robot's state transition between the previous and next two frames on the image plane; the output of the neural network scoring model is a score value between 0 and 1; for the state transition sampled from the example data distribution, the model is trained to output 1; and for the state transition recorded during the robot's motion, the model is trained to output 0; the neural network scoring model makes judgments in the image space; including the projection coordinates of the robot joints on the image plane or the optical flow information extracted from the image; the scoring model is implemented using a fully connected neural network, and the input is a 52-dimensional observation vector, which contains the image projection coordinates (13*2) of each joint of the robot and the optical flow information (13*2) at the projection. After several layers of fully connected layers, a tensor with a dimension of 1 is input, and after the sigmoid() activation function, a score between 0 and 1 is output.

[0072] In one embodiment, S400 includes:

[0073] S401, the agent learning module coordinately optimizes the robot control strategy and the camera movement strategy based on the feedback of the scoring result of the motion video recording data of the neural network scoring model through the reward feedback of the neural network scoring model, and obtains the optimized robot control strategy and the optimized camera movement strategy;

[0074] S402, iteratively optimize the agent strategy according to the optimized robot control strategy and the optimized camera movement strategy, and update it to the virtual robot and virtual camera combined agent.

[0075] The principle and effect of the above technical solution are as follows: the intelligent agent learning module is based on the feedback of the scoring results of the motion video recording data of the neural network scoring model, and the robot control strategy and the camera movement strategy are collaboratively optimized through the reward feedback of the neural network scoring model to obtain the optimized robot control strategy and the optimized camera movement strategy; according to the optimized robot control strategy and the optimized camera movement strategy, the intelligent agent strategy is iteratively optimized and updated to the virtual robot and virtual camera combined intelligent agent;

[0076] Optimize the robot control strategy and the camera movement strategy, iteratively optimize the agent strategy, and update it to the virtual robot and virtual camera combined agent, including: introducing reward signals in the initial stage of optimizing the robot control strategy and optimizing the camera movement strategy to assist algorithm convergence; achieving more robust training results; defining the policy network and reward function; using the reinforcement learning algorithm to update the policy network; the updated camera movement strategy and robot control strategy are used for a new round of data collection and iterative training, thereby iteratively optimizing the agent strategy; defining the policy network and reward function include: preventing the robot from falling, limiting the robot's range of motion, and ensuring that the camera can always record the robot's full body or partial torso rewards.

[0077] The present invention provides a multi-source video data robot skill system, comprising:

[0078] The video collection data expansion subsystem automatically collects sample videos related to the skill and performs data expansion based on the text description of the sport skill through the sample video collection module to obtain the sport sample video data;

[0079] Virtually construct the control strategy subsystem, build and instantiate the virtual robot and virtual camera, coordinate the robot control strategy and camera movement strategy and combine the intelligent body, generate and record the robot motion video recording data;

[0080] The sports skill intelligent scoring subsystem builds a video intelligent scoring model through the sports skill video scoring module to generate scoring results for the robot's sports video recording data;

[0081] The strategy optimization agent update subsystem sets the reward feedback of the neural network scoring model through the agent learning module to collaboratively optimize the robot control strategy and camera movement strategy, and updates them to the agent.

[0082] The principle and effect of the above technical solution are as follows: the present invention provides a multi-source video data robot skill system, including: a video collection data expansion subsystem, which automatically collects sample videos related to the skill and performs data expansion according to the text description of the sports skill through the sample video collection module, and obtains sports sample video data; a virtual construction control strategy subsystem, which constructs and instantiates a virtual robot and a virtual camera, coordinates the robot control strategy with the camera movement strategy and combines the intelligent body to generate and record the robot motion video recording data; a sports skill intelligent scoring subsystem, which constructs a video intelligent scoring model through the sports skill video scoring module, and generates a scoring result for the robot motion video recording data; a strategy optimization intelligent body update subsystem, which sets a neural network scoring through the intelligent body learning module The reward feedback of the model collaboratively optimizes the robot control strategy and camera movement strategy, and updates them to the intelligent agent; this patent proposes a skill learning method based on multi-source video data, aiming to enable the virtual robot to learn a unified motion control strategy from multi-source video examples; this method not only trains the robot's control strategy, but also trains the camera movement strategy to record the robot, and uses the recording results to score the actions to assist its learning; this method does not need to restore the three-dimensional motion trajectory in the example video for training, which significantly reduces the complexity of data acquisition and preprocessing, thereby effectively utilizing multi-source video data for learning; this method effectively avoids the data heterogeneity problem caused by different sources of training data, enabling the virtual robot to learn unified motion skill keywords from massive example videos from different sources. This method simplifies the complex process of traditional methods in the training data collection and preprocessing stages, and improves the processing efficiency of training data; multi-source video data type; this method models the skill learning problem based on multi-source video data as a multi-agent reinforcement learning problem. While training the robot motion control strategy, it also trains the camera movement strategy to assist the robot in completing skill learning; the camera movement strategy trained by this method does not require human control, and can adjust its own recording trajectory according to the robot's motion trajectory, naturally showing the skill learning results of the virtual robot; a method for learning unified motion skill keywords from multi-source video data; a method for jointly training robot motion control and camera movement strategies; a motion skill keyword learning method based on multi-source video data is implemented. This method can use multi-source motion skill keyword videos collected from the Internet to supervise the virtual robot to obtain new motion skill keywords through imitation learning; since this method does not require the collection of training videos in a specific way, it can use massive network data to learn diverse skills; in addition, this method can also generate a natural camera movement strategy while training the robot to learn motion skill keywords. This strategy can record motion videos for the robot to intuitively show its learning results; such as Figure 1As shown, the example video collection module collects motion example videos through big data tag content capture, and analyzes and determines the motion keyword information of the motion example videos; the motion keyword information includes: motion description keywords and motion action feature keywords; the motion description keywords perform video text description analysis through the text description content of the motion example videos and the subtitle text description content to obtain the video text description analysis results; and compare the video text description analysis results with the motion action feature keywords to determine whether the video text description and the motion action features are consistent; if the video text description and the motion action features are consistent, the motion example video data is transmitted to the skill video scoring module; the skill video scoring module receives the 3D simulation data in parallel. The skill video recording module in the environment system records the robot motion video recording data; the virtual robot and the virtual camera are combined to construct a combined intelligent body, the combined intelligent body records the virtual robot video, and obtains the robot motion video recording data; the robot motion video recording data is transmitted to the skill video scoring module; the skill video scoring module scores the motion example video data and the robot motion video recording data, and obtains the robot motion video recording data scoring information; the robot motion video recording data scoring information is transmitted to the intelligent body learning module for intelligent learning training; the intelligent learning training information is continuously transmitted to the 3D simulation environment, and the 3D simulation environment and the combined intelligent body are coordinated and optimized according to the intelligent learning training information.

[0083] In one embodiment, the video collection data expansion subsystem includes:

[0084] The keyword extraction and parsing unit sets a sports skill text description; the sports skill keywords in the sports skill text description are parsed and extracted from the sports skill text description by using a keyword extraction algorithm combined with a large language model;

[0085] A tag content aggregation unit aggregates tag content of sports skill keywords to obtain sports skill tags;

[0086] The video tag expansion unit collects sports example videos and performs data expansion based on sports skill keywords and sports skill tags to obtain sports example video data and sports video expansion data.

[0087] The principle and effect of the above technical solution are as follows: the video collection data expansion subsystem includes: a keyword extraction and parsing unit, which sets a sports skill text description; through a keyword extraction algorithm, combined with a large language model, the sports skill keywords in the sports text description are parsed and extracted; a tag content aggregation unit performs tag content aggregation on the sports skill keywords to obtain sports skill tags; a video tag collection expansion unit collects sports example videos and performs data expansion according to sports skill keywords and sports skill tags to obtain sports example video data and sports video expansion data; according to sports skill keywords and sports skill tags, collects sports example videos and performs data expansion to obtain sports example video data and sports video expansion data, including: according to sports skill keywords and sports skill tags, collects example videos through data crawling to obtain collected sports example videos; and matches sports skill tags with reference sports tags in existing sports data sets, takes out sports videos with a matching degree higher than the set reference matching degree for data expansion, and obtains sports video expansion data; transmits the sports example video data and sports video expansion data to the skill video scoring module; such as Figure 2 As shown in the figure, by analyzing the text of the skill description, sample video data that meets the description is collected; a simulation environment is constructed and the robot and camera are instantiated, and the camera movement and control strategies are initialized / updated; the robot's motion video is recorded according to the camera movement strategy, and a scoring model is trained in combination with the sample video; the agent strategy is learned and optimized based on the feedback of the scoring model, and updated to the simulation environment.

[0088] In one embodiment, a virtual construction control strategy subsystem includes:

[0089] The robot structure modeling unit uses the simulation experiment platform to perform robot virtual modeling according to the robot parameter information, and constructs a virtual robot composed of multiple rigid bodies.

[0090] A virtual camera modeling unit performs camera modeling according to camera parameter information, constructs a pinhole model camera with a collision-free property, and obtains a virtual camera;

[0091] Instantiate the intelligent agent combination unit, instantiate the virtual robot and the virtual camera, coordinate the robot control strategy with the camera movement strategy, and combine the instantiated virtual robot and the virtual camera into an intelligent agent to control the virtual robot and record motion video, generate and record the robot motion video captured by the camera in the simulation environment, and obtain the robot motion video recording data; transmit the robot motion video recording data to the sports skill video scoring module.

[0092] The principle and effect of the above technical solution are as follows: a virtual construction control strategy subsystem includes: a robot structure modeling unit, which uses a simulation experiment platform to perform robot virtual modeling according to robot parameter information, and constructs a virtual robot with a virtual structure composed of multiple rigid bodies; a virtual camera modeling unit, which performs camera modeling according to camera parameter information, constructs a pinhole model camera with a collision-free attribute, and obtains a virtual camera; an instantiated intelligent body combination unit, which instantiates the virtual robot and the virtual camera, coordinates the robot control strategy with the camera movement strategy, and combines the instantiated virtual robot and the virtual camera into an intelligent body, performs virtual robot control and motion video recording, generates and records the robot motion video captured by the camera in the simulation environment, and obtains the robot motion video recording data; transmits the robot motion video recording data to the motion skill video scoring module; uses a simulation experiment platform to perform robot virtual modeling according to robot parameter information, and constructs a virtual robot with a virtual structure composed of multiple rigid bodies. 211 includes: using a simulation experiment platform to construct a virtual structure composed of multiple rigid bodies according to robot parameter information, and each virtual structure The parts are connected by virtual joints; the knee joint and elbow joint use 1-DOF rotational joints; the other joints are set as 3-DOF spherical joints; the robot character includes 13 joints and 34 degrees of freedom; set the robot entity virtual association parameters; the robot entity virtual association parameters include: the total mass of the robot is 45 kg; the robot height is 1.62 meters; the robot state space dimension is 197 dimensions; the robot action space dimension is 36 dimensions; build a robot control strategy network model; building a robot control strategy network model includes: taking the robot state information as input, and outputting the torque value used to control each joint; the robot state information includes the spatial position and movement speed of each joint of the robot; control strategy modeling is performed through a fully connected neural network, with input and output dimensions of 197 and 36 respectively, to obtain the robot control strategy network model; the training scene is a randomly initialized ground, and the robot makes different actions on the ground; the ground is set with multiple grid types and multiple physical parameters during initialization to improve the robustness of the robot control strategy; the multiple grid types include: flat grid and rugged grid; the multiple physical parameters include: friction coefficient and elastic recovery coefficient;

[0093] According to the camera parameter information, camera modeling is performed to construct a pinhole model camera with collision-free properties, and obtaining a virtual camera 212 includes: randomly sampling and generating the camera's internal parameter parameters within a certain range to cover the camera's internal parameter range used when collecting multi-source video data; constructing a camera movement strategy network model; the input of the camera movement strategy network model includes the world poses of the camera and the robot in the past several frames, the relative poses of the camera and the robot, and the pixel coordinates of the robot's joints projected in the picture; the output of the camera movement strategy network model is the camera's next movement instruction, and the next movement instruction is decomposed into a rotation instruction and a displacement instruction; the virtual The robot and the virtual camera are instantiated, the robot control strategy and the camera movement strategy are coordinated, and the instantiated virtual robot and the virtual camera are combined into an intelligent entity to control the virtual robot and record motion videos, including: the robot and the camera are instantiated for the first time, and the parameters of the robot control strategy network model and the camera movement strategy network model are randomly initialized; in the subsequent learning process, the strategy networks of the robot control strategy network model and the camera movement strategy network model are gradually optimized and updated; the robot control strategy network model enables the robot to accurately reproduce the action skills in the example video, and the camera movement strategy network model records the motion video from an angle similar to the example video.

[0094] In one embodiment, the sports skill intelligent scoring subsystem includes:

[0095] A feature extraction and storage unit is used to extract and store features of robot motion video recording data, robot motion example video data, and motion video expansion data;

[0096] The scoring model feature scoring unit constructs a neural network scoring model, compares the feature similarity between the robot motion video recording data and the robot motion example video data and the motion video expansion data, and generates the motion video recording data scoring result.

[0097] The principle and effect of the above technical solution are as follows: the sports skill intelligent scoring subsystem includes: a feature extraction storage unit, which extracts and stores features of robot motion video recording data, robot motion example video data and motion video expansion data; a scoring model feature scoring unit, which constructs a neural network scoring model, compares the feature similarity of robot motion video recording data with robot motion example video data and motion video expansion data, and generates a scoring result of the motion video recording data; constructs a neural network scoring model, and compares the feature similarity of robot motion video recording data with robot motion example video data and motion video expansion data, including: the input of the neural network scoring model is the projection feature of the robot's state transition between the previous and next two frames on the image plane; the neural network scoring model is used to generate a scoring result of the motion video recording data ... The output of the network scoring model is a score value between 0 and 1; for state transitions sampled from the sample data distribution, the model is trained to output 1; and for state transitions recorded during robot motion, the model is trained to output 0; the neural network scoring model makes judgments in image space; including the projection coordinates of the robot joints on the image plane or the optical flow information extracted from the image; the scoring model is implemented using a fully connected neural network, with an input of a 52-dimensional observation vector, containing the image projection coordinates (13*2) of each joint of the robot and the optical flow information (13*2) at the projection. After several layers of fully connected layers, a tensor of dimension 1 is input, and after a sigmoid() activation function, a score between 0 and 1 is output.

[0098] In one embodiment, the policy optimization agent update subsystem includes:

[0099] The intelligent agent learning optimization unit, the intelligent agent learning module is based on the feedback of the scoring results of the motion video recording data of the neural network scoring model, and the reward feedback of the neural network scoring model is used to coordinately optimize the robot control strategy and the camera movement strategy to obtain the optimized robot control strategy and the optimized camera movement strategy;

[0100] The intelligent agent strategy iteration and updating unit iteratively optimizes the intelligent agent strategy according to the optimized robot control strategy and the optimized camera movement strategy, and updates it to the virtual robot and virtual camera combined intelligent agent.

[0101] The principle and effect of the above technical solution are as follows: the strategy optimization agent update subsystem includes: an agent learning optimization unit, the agent learning module is based on the feedback of the scoring results of the motion video recording data of the neural network scoring model, and the robot control strategy and the camera movement strategy are collaboratively optimized through the reward feedback of the neural network scoring model to obtain the optimized robot control strategy and the optimized camera movement strategy; the agent strategy iteration update unit, according to the optimized robot control strategy and the optimized camera movement strategy, iteratively optimizes the agent strategy and updates it to the virtual robot and virtual camera combination agent; optimizes the robot control strategy and the optimized camera movement strategy, iteratively optimizes the agent strategy The next generation of optimized agent strategies are updated to the virtual robot and virtual camera combined agents, including: introducing reward signals in the initial stage of optimizing the robot control strategy and optimizing the camera movement strategy to assist the algorithm convergence; achieving more robust training results; defining the strategy network and reward function; using the reinforcement learning algorithm to update the strategy network; the updated camera movement strategy and robot control strategy are used for a new round of data collection and iterative training, thereby iteratively optimizing the agent strategy; defining the strategy network and reward function include: preventing the robot from falling, limiting the robot's range of motion, and ensuring that the camera can always record the robot's full body or partial torso rewards.

[0102] Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the specification and the implementation methods. They can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and the illustrations shown and described herein.

Claims

1. A robot skill learning method based on multi-source video data, characterized in that: include: S100, automatically collecting sample videos related to the skill according to the text description of the sport skill through a sample video collection module and performing data expansion to obtain sport sample video data; S200, constructing and instantiating a virtual robot and a virtual camera, coordinating a robot control strategy and a camera movement strategy and combining intelligent agents, and generating and recording robot motion video recording data; S300, through the sports skill video scoring module, builds a video intelligent scoring model to generate scoring results for the robot sports video recording data; S400, through the agent learning module, sets the reward feedback of the neural network scoring model to collaboratively optimize the robot control strategy and camera movement strategy, and updates them to the agent; S200 includes: S201, using the simulation experiment platform, based on the robot parameter information, performing robot virtual modeling, and constructing a virtual robot composed of a plurality of rigid bodies and a virtual structure; S202, performing camera modeling according to the camera parameter information, constructing a pinhole model camera with a collision-free property, and obtaining a virtual camera; S203, instantiating the virtual robot and the virtual camera, coordinating the robot control strategy with the camera movement strategy, and combining the instantiated virtual robot and the virtual camera into an intelligent body, performing virtual robot control and motion video recording, generating and recording the robot motion video captured by the camera in the simulation environment, and obtaining the robot motion video recording data; transmitting the robot motion video recording data to the sports skill video scoring module; The control strategy is modeled through a fully connected neural network to obtain the robot control strategy network model; A camera movement strategy network model is constructed; the input of the camera movement strategy network model includes the world poses of the camera and the robot in the past several frames, the relative poses of the camera and the robot, and the pixel coordinates of the robot joints projected in the picture; the output of the camera movement strategy network model is the next camera movement instruction, and the next camera movement instruction is decomposed into a camera rotation instruction and a camera displacement instruction; the virtual robot and the virtual camera are instantiated, the robot control strategy and the camera movement strategy are coordinated, and the instantiated virtual robot and the virtual camera are combined into an intelligent body, and the virtual robot control and motion video recording are performed, including: the robot and the camera are instantiated for the first time, and the parameters of the robot control strategy network model and the camera movement strategy network model are randomly initialized; in the subsequent learning process, the strategy networks of the robot control strategy network model and the camera movement strategy network model are gradually optimized and updated; the robot control strategy network model enables the robot to accurately reproduce the action skills in the example video, and the camera movement strategy network model records the motion video from an angle similar to the example video.

2. The method for learning robot skills from multi-source video data according to claim 1, characterized in that: S100 includes: S101, setting a sports skill text description; parsing and extracting sports skill keywords from the sports skill text description by using a keyword extraction algorithm in combination with a large language model; S102, performing tag content aggregation on sports skill keywords to obtain sports skill tags; S103, according to the sports skill keywords and sports skill tags, collect sports example videos and perform data expansion to obtain sports example video data and sports video expansion data.

3. The method for learning robot skills from multi-source video data according to claim 1, characterized in that: S300 includes: S301, extracting and storing features of robot motion video recording data, robot motion example video data, and motion video expansion data; S302, constructing a neural network scoring model, comparing the feature similarity between the robot motion video recording data and the robot motion example video data and the motion video expansion data, and generating a scoring result for the motion video recording data.

4. The method for learning robot skills from multi-source video data according to claim 1, characterized in that: S400 includes: S401, the agent learning module coordinately optimizes the robot control strategy and the camera movement strategy based on the feedback of the scoring result of the motion video recording data of the neural network scoring model through the reward feedback of the neural network scoring model, and obtains the optimized robot control strategy and the optimized camera movement strategy; S402, iteratively optimize the agent strategy according to the optimized robot control strategy and the optimized camera movement strategy, and update it to the virtual robot and virtual camera combined agent.

5. A multi-source video data robot skill learning system, characterized in that: include: The video collection data expansion subsystem automatically collects sample videos related to the skill and performs data expansion based on the text description of the sport skill through the sample video collection module to obtain the sport sample video data; Virtually construct the control strategy subsystem, build and instantiate the virtual robot and virtual camera, coordinate the robot control strategy and camera movement strategy and combine the intelligent body, generate and record the robot motion video recording data; The sports skill intelligent scoring subsystem builds a video intelligent scoring model through the sports skill video scoring module to generate scoring results for the robot's sports video recording data; The strategy optimization agent update subsystem uses the agent learning module to set the reward feedback of the neural network scoring model to collaboratively optimize the robot control strategy and camera movement strategy, and update them to the agent; Virtual construction control strategy subsystem, including: The robot structure modeling unit uses the simulation experiment platform to perform robot virtual modeling according to the robot parameter information, and constructs a virtual robot composed of multiple rigid bodies. A virtual camera modeling unit performs camera modeling according to camera parameter information, constructs a pinhole model camera with a collision-free property, and obtains a virtual camera; Instantiate the intelligent agent combination unit, instantiate the virtual robot and the virtual camera, coordinate the robot control strategy with the camera movement strategy, and combine the instantiated virtual robot and virtual camera into an intelligent agent to control the virtual robot and record motion video, generate and record the robot motion video captured by the camera in the simulation environment, and obtain the robot motion video recording data; transmit the robot motion video recording data to the sports skill video scoring module; The control strategy is modeled through a fully connected neural network to obtain the robot control strategy network model; A camera movement strategy network model is constructed; the input of the camera movement strategy network model includes the world poses of the camera and the robot in the past several frames, the relative poses of the camera and the robot, and the pixel coordinates of the robot joints projected in the picture; the output of the camera movement strategy network model is the next camera movement instruction, and the next camera movement instruction is decomposed into a camera rotation instruction and a camera displacement instruction; the virtual robot and the virtual camera are instantiated, the robot control strategy and the camera movement strategy are coordinated, and the instantiated virtual robot and the virtual camera are combined into an intelligent body, and the virtual robot control and motion video recording are performed, including: the robot and the camera are instantiated for the first time, and the parameters of the robot control strategy network model and the camera movement strategy network model are randomly initialized; in the subsequent learning process, the strategy networks of the robot control strategy network model and the camera movement strategy network model are gradually optimized and updated; the robot control strategy network model enables the robot to accurately reproduce the action skills in the example video, and the camera movement strategy network model records the motion video from an angle similar to the example video.

6. A multi-source video data robot skill learning system according to claim 5, characterized in that: Video collection data expansion subsystem, including: The keyword extraction and parsing unit sets a sports skill text description; the sports skill keywords in the sports skill text description are parsed and extracted from the sports skill text description by using a keyword extraction algorithm combined with a large language model; A tag content aggregation unit aggregates tag content of sports skill keywords to obtain sports skill tags; The video tag expansion unit collects sports example videos and performs data expansion based on sports skill keywords and sports skill tags to obtain sports example video data and sports video expansion data.

7. The multi-source video data robot skill learning system according to claim 5, characterized in that: Sports skills intelligent scoring subsystem, including: A feature extraction and storage unit is used to extract and store features of robot motion video recording data, robot motion example video data, and motion video expansion data; The scoring model feature scoring unit constructs a neural network scoring model, compares the feature similarity between the robot motion video recording data and the robot motion example video data and the motion video expansion data, and generates the motion video recording data scoring result.

8. The multi-source video data robot skill learning system according to claim 5, characterized in that: The policy optimization agent update subsystem includes: The intelligent agent learning optimization unit, the intelligent agent learning module is based on the feedback of the scoring results of the motion video recording data of the neural network scoring model, and the reward feedback of the neural network scoring model is used to coordinately optimize the robot control strategy and the camera movement strategy to obtain the optimized robot control strategy and the optimized camera movement strategy; The intelligent agent strategy iteration and updating unit iteratively optimizes the intelligent agent strategy according to the optimized robot control strategy and the optimized camera movement strategy, and updates it to the virtual robot and virtual camera combined intelligent agent.

Images