Robot competitive motion control method and system based on motion database

Abstract

The invention discloses a robot competitive motion control method and system based on a motion database. The robot competitive motion control method comprises the following steps of S1, target trajectory prediction; s2, training a steady-state motion network; s3, establishing a steady-state control strategy; s4, establishing a skill control strategy; and S5, switching the strategy network. The multi-modal strategy provided by the invention is used as an alternative scheme before a proper normal form appears, the problem that a unified strategy is difficult to train can be solved, special skills are learned through expert data, the integrity of a single expert action network can be reserved, the success rate of actions can be improved, the debugging difficulty can be reduced, and a steady-state control strategy based on a database can be used for realizing multi-modal control. The database framework has expansibility, the dynamic switching mechanism has high adaptability to burst interference, and the robustness of the system can be enhanced.

Description

Technical Field

[0001] This invention belongs to the field of robot control, specifically relating to a robot competitive sports control method and system based on a motion database. Background Technology

[0002] Neural networks, with their superior robustness and nonlinear fitting capabilities, are gradually replacing traditional model-based control methods as the mainstream solution in robot control. However, their black-box nature and convergence issues make it difficult to train alternating behaviors for multiple complex tasks. For example, in competitive sports like basketball, which involves alternating tasks such as dribbling, catching, and shooting, training with a single network is unlikely to achieve ideal convergence, and distillation techniques can easily lose knowledge of the teacher's strategy.

[0003] Inspired by the concept of "special move databases" in video games, this invention proposes a scheme based on a motion strategy database. It obtains relatively general steady-state motion strategies through reinforcement learning, adaptable to most competitive sports scenarios. Furthermore, it extracts the movements of real expert players through imitation learning to obtain a skill database. Based on suitable conditions, it selects appropriate skill strategies to replace the steady-state strategies and switches back to the steady-state strategy after the action ends. Summary of the Invention

[0004] The technical problem to be solved by this invention is to realize a dynamic switching mechanism of strategies based on environmental state and multimodal strategy architecture to meet the control requirements of robots in complex tasks, thereby reducing the difficulty of debugging and improving the success rate of actions.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a robot competitive motion control method based on a motion database, comprising the following steps: Step S1: Target trajectory prediction; A target trajectory tracker is obtained by capturing target images using a binocular camera, extracting target features using a neural network, and updating the model using online learning or domain adaptation techniques. Step S2: Steady-state motion network training; Training a conventional motion network enables the robot to move and perform target tasks. This conventional motion network includes policy networks and other networks developed through reinforcement learning that can achieve the robot's target tasks. Step S3: Establish a steady-state control strategy; The conventional network trained in step S2 is deployed, and combined with aerodynamics, the target trajectory predicted in step S1 is corrected. The action execution point closest to the current robot torso position is generated, and the conventional network is used to complete the robot's specific actions in a specific task. The trajectory of the target object and the conventional network together form a steady-state control strategy for the robot in complex tasks. Step S4: Establish a skill control strategy; Expert motion data corresponding to the target task is collected. Based on this expert motion data, the data is redirected to the robot's reference state. Through imitation learning, the robot is given style rewards for individual skills. Combined with task rewards, a policy network database with multiple specialized skills is obtained. The imitation learning is only performed on skill-based tasks excluding regular sub-tasks. Expert data is collected specifically for skill-based sub-tasks within the target task. Step S5: Switch the policy network; Design a switching logic and implement the switching of multiple policy networks using a state machine. The policy network selection is achieved automatically based on the state machine, and this automatic switching logic can be implemented through methods such as training neural networks and mathematical statistics. Different state machines are obtained by dividing the states according to the specific task. Multiple policy networks can contain two or more policy networks, and the selection of policy networks is determined by the nature of the task. Different policy networks are responsible for the policy output of different subtasks. Taking the automatic switching logic of two policy networks implemented by training a neural network as an example, its selection can be expressed as: ; Furthermore, the online learning content includes target tracking based on Kalman filtering and neural networks. Domain adaptation technology refers to obtaining the target trajectory through cross-domain feature alignment and transfer learning based on features extracted by neural networks. Target tracking based on Kalman filtering and neural networks uses neural networks to predict the target trajectory at discontinuous time points, which is used to correct the target trajectory obtained by Kalman filtering. Kalman filtering uses a dynamic model to correct the observation error of the neural network.

[0006] Furthermore, it includes a target trajectory prediction module, a steady-state motion network training module, a steady-state control strategy construction module, a skill control strategy construction module, and a strategy network switching module; the target trajectory prediction module is used for predicting the trajectory of the target; the steady-state motion network training module is used for training the robot's movement and target tasks; the steady-state control strategy construction module is used for acquiring the robot's strategy in regular tasks; the skill control strategy construction module is used for acquiring the robot's strategy in skill tasks; and the strategy network switching module is used for automatically selecting the strategy network under different task states.

[0007] Furthermore, the target trajectory prediction module includes an online learning module or a domain adaptation technology module, both of which are used to track the target trajectory.

[0008] Furthermore, the online learning module includes a Kalman filter module.

[0009] Furthermore, the domain adaptation technology module includes cross-domain feature alignment and transfer learning modules.

[0010] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: 1. The multimodal strategy proposed in this invention serves as an alternative before a suitable paradigm emerges, and can solve the problem of the difficulty in training a unified strategy.

[0011] 2. This invention learns specialized skills using expert data, which can preserve the integrity of individual expert action networks, improve the success rate of actions, and reduce debugging difficulty.

[0012] 3. The database-based steady-state control strategy proposed in this invention has a database framework that is easily scalable.

[0013] 4. The dynamic switching mechanism proposed in this invention has strong adaptability to sudden interference and can enhance the robustness of the system. Attached Figure Description

[0014] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings.

[0015] Figure 1 This is a schematic diagram of robot competitive sports control based on a motion database.

[0016] Figure 2 This is a flowchart of the overall control process of the present invention. Detailed Implementation

[0017] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Example 1: The following describes a robot competitive motion control method based on a motion database provided by the present invention, including the following steps: Step S1: Target trajectory prediction; A target trajectory tracker based on a binocular camera is designed, using a convolutional neural network (CNN) to extract target features and updating the model through online learning or domain adaptation techniques. Common target tracking algorithms such as YOLO and SiamRPN are cascaded to a lower-level extended Kalmann filter to denoise the observed trajectory. For example, when the ball is obstructed by a net, the trajectory can be predicted. Step S2: Steady-state motion network training; Training a conventional motion network enables the robot to move and perform target tasks. This conventional motion network includes policy networks from reinforcement learning, which can achieve the robot's target tasks. Specifically, using badminton as an example, in the training environment, the training target is the random swing point where the racket tip passes at a given moment, and a fall penalty is applied. This trains the robot's movement and receiving abilities. Step S3: Establish a steady-state control strategy; The conventional network trained in step S2 is deployed, and combined with aerodynamics, the predicted target trajectory in step S1 is corrected. The action execution point closest to the current robot torso position is generated. The conventional network is used to complete the robot's specific actions in a particular task. The trajectory of the target object and the conventional network together form a steady-state control strategy for the robot in complex tasks. Specifically, combined with aerodynamics, the ball trajectory is predicted, and the receiving point closest to the current robot torso position is generated. Based on the conventional network, the robot's badminton competition control strategy is completed. Step S4: Establish a skill control strategy; Expert motion data corresponding to the target task is collected. Based on this expert motion data, the data is redirected to the robot's reference state. Through imitation learning, the robot is given rewards for specific skill styles, such as serving and spiking. Combined with task rewards, such as randomly selecting a certain range as the target receiving point position during spiking style training, a strategy network database with multiple specialized skills is obtained. The imitation learning is only performed on skill-based tasks other than regular sub-tasks. Expert data is collected for skill-based sub-tasks within the target task. Step S5: Switch the policy network; Design switching logic and implement the switching of multiple policy networks using a state machine. For example, when the trajectory of a ball rises to a threshold, it is considered a state where the "snap" policy is switched to, replacing the original conventional policy as the system output, thus achieving robot motion control. The policy network is selected automatically based on the state machine. The automatic switching logic can be implemented through methods such as training neural networks and mathematical statistics. Different state machines are obtained by dividing the states according to the specific task. Multiple policy networks can contain two or more policy networks, and the selection of policy networks is determined by the nature of the task. Different policy networks are responsible for the policy output of different sub-tasks. Taking the automatic logic switching of two policy networks through neural network training as an example, its selection can be expressed as: ; The online learning content includes target tracking based on Kalman filtering and neural networks. Domain adaptation technology refers to obtaining target trajectories through cross-domain feature alignment and transfer learning based on features extracted by neural networks. Target tracking based on Kalman filtering and neural networks uses neural networks to predict target trajectories at discontinuous time points to correct the target trajectories obtained by Kalman filtering. Kalman filtering uses dynamic models to correct the observation errors of neural networks.

[0019] A robot competitive motion control system based on a motion database includes a target trajectory prediction module, a steady-state motion network training module, a steady-state control strategy construction module, a skill control strategy construction module, and a strategy network switching module. The target trajectory prediction module is used for predicting the trajectory of the target. The steady-state motion network training module is used to train the robot's movement and target tasks. The steady-state control strategy construction module is used to acquire the robot's strategy in regular tasks. The skill control strategy construction module is used to acquire the robot's strategy in skill-based tasks. The strategy network switching module is used to automatically select the strategy network for different task states.

[0020] The main system components for the overall control of robotic competitive sports based on motion databases include: sensor group, processor, and actuator group; The sensor group (camera, inertial measurement unit IMU, lidar, encoder, etc.) collects the competition environment signals in real time (such as images of opponents / balls, field area images, and its own status), and transmits the signals to the processor through the communication bus (such as CAN, I2C); The processor main control board integrates CPU / GPU / edge computing units, responsible for processing sensor signals, running motion databases, strategy selection algorithms (neural networks), and control logic. The processor perception layer first processes the sensor signals as in step S1. The decision layer processor calls the attitude data from the motion database and runs the neural network model (special strategy / conventional strategy) to complete the strategy selection step S5. Finally, the control layer performs strategy inference and outputs the final control commands (such as motor current signals and servo angles). The actuator group consists of physical components (servo motors, motor controllers, servo motors, servo motor controllers) that drive the robot's movement, translating strategy commands into mechanical actions; Among them, the five main modules of target trajectory prediction, steady-state motion network training, steady-state control strategy construction, skill control strategy construction and strategy network switching are based on the signals of the sensor components. The functions of each module are completed in the processor main control board, and the final action is sent through the actuator components.

[0021] The target trajectory prediction module includes an online learning module or a domain adaptation technology module, both of which are used to track the target trajectory.

[0022] The online learning module includes a Kalman filter module.

[0023] The domain adaptation technology module includes cross-domain feature alignment and transfer learning modules.

[0024] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to any specific implementation. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for controlling robot competitive motion based on a motion database, characterized in that, Includes the following steps: Step S1: Target trajectory prediction; A target trajectory tracker is obtained by capturing target images using a binocular camera, extracting target features using a neural network, and updating the model using online learning or domain adaptation techniques. Step S2: Steady-state motion network training; Training a conventional motion network enables the robot to move and perform target tasks. This conventional motion network includes policy networks and other networks developed through reinforcement learning that can achieve the robot's target tasks. Step S3: Establish a steady-state control strategy; The conventional network trained in step S2 is deployed, and combined with aerodynamics, the target trajectory predicted in step S1 is corrected. The action execution point closest to the current robot torso position is generated, and the conventional network is used to complete the robot's specific actions in a specific task. The trajectory of the target object and the conventional network together form a steady-state control strategy for the robot in complex tasks. Step S4: Establish a skill control strategy; Expert motion data corresponding to the target task is collected. Based on this expert motion data, the data is redirected to the robot's reference state. Through imitation learning, the robot is given style rewards for individual skills. Combined with task rewards, a policy network database with multiple specialized skills is obtained. The imitation learning is only performed on skill-based tasks excluding regular sub-tasks. Expert data is collected specifically for skill-based sub-tasks within the target task. Step S5: Switch the policy network; Design a switching logic and implement the switching of multiple policy networks using a state machine. The policy network selection is achieved automatically based on the state machine, and this automatic switching logic can be implemented through methods such as training neural networks and mathematical statistics. Different state machines are obtained by dividing the states according to the specific task. Multiple policy networks can contain two or more policy networks, and the selection of policy networks is determined by the nature of the task. Different policy networks are responsible for the policy output of different subtasks. Taking the automatic switching logic of two policy networks implemented by training a neural network as an example, its selection can be expressed as: 。 2. The robot competitive motion control method based on a motion database according to claim 1, characterized in that: The online learning content includes target tracking based on Kalman filtering and neural networks. Domain adaptation technology refers to obtaining target trajectories through cross-domain feature alignment and transfer learning based on features extracted by neural networks. Target tracking based on Kalman filtering and neural networks uses neural networks to predict target trajectories at discontinuous time points to correct the target trajectories obtained by Kalman filtering. Kalman filtering uses dynamic models to correct the observation errors of neural networks.

3. A robot competitive sports control system based on a motion database, characterized in that, The system includes a target trajectory prediction module, a steady-state motion network training module, a steady-state control strategy construction module, a skill control strategy construction module, a strategy network switching module, a control terminal, and a processor. The target trajectory prediction module is used for predicting the trajectory of the target. The steady-state motion network training module is used to train the robot's movement and target tasks. The steady-state control strategy construction module is used to acquire the robot's strategy in regular tasks. The skill control strategy construction module is used to acquire the robot's strategy in skill tasks. The strategy network switching module is used to automatically select the strategy network for different task states.

4. The robot competitive sports control system based on a motion database according to claim 3, wherein the target trajectory prediction module includes an online learning module or a domain adaptation technology module, and both are used to track the target trajectory.

5. The robot competitive sports control system based on a motion database according to claim 4, wherein the online learning module includes a Kalman filter module.

6. The robot competitive sports control system based on a motion database according to claim 4, wherein the domain adaptation technology module includes a cross-domain feature alignment and transfer learning module.

Images