A control method for musculoskeletal robots in table tennis scenarios

Abstract

This invention discloses a musculoskeletal robot control method for table tennis rallies. The method includes: at the beginning of each stroke cycle, using a trajectory planner based on a physics model to predict the trajectory of the table tennis ball, and calculating the expected hitting position, hitting time, hitting speed, and hitting direction of the racket held by the musculoskeletal robot based on a preset hitting plane and target landing point, forming a high-level control command; inputting the high-level control command and the current environmental observation state into a pre-trained reinforcement learning policy network, outputting the target joint position of the musculoskeletal robot; and using an adaptive controller based on forward kinematics to map the target joint position to the activation signals of each muscle, driving the musculoskeletal robot to perform actions. This invention solves the control problem of high-dimensional musculoskeletal systems in complex dynamic tasks through a hierarchical architecture, achieving continuous table tennis rallies with high success rate and low muscle metabolic cost.

Description

Technical Field

[0001] This invention belongs to the field of robot control and artificial intelligence technology, and specifically relates to a musculoskeletal robot control method for table tennis match scenarios. Background Technology

[0002] Musculoskeletal robots mimic the muscle-driven characteristics of humans, possessing flexibility and the potential for high explosive power, but their control faces significant challenges. These systems typically have extremely high dimensionality (e.g., more than 270 muscles), exhibiting severe actuation redundancy and nonlinear dynamics. Researching such systems helps reveal the hierarchical organizational principles of biological motion control, such as how the nervous system separates task planning from biomechanical execution. The research results can be transferred and applied to the assistive control of neurorehabilitation robots, aiding in understanding how desired joint movements can be achieved through the activation of specific muscle groups.

[0003] Traditional control methods are difficult to apply directly to such complex systems. In recent years, end-to-end reinforcement learning has been used to solve musculoskeletal control problems, but directly mapping observations to the activation of hundreds of muscles often leads to problems such as unstable training, low sample efficiency, and difficulty in generating coordinated movements.

[0004] Especially in table tennis rallies, robots need to precisely control the position and speed of the racket at extremely high speeds to achieve continuous multi-round rallies. This tests the robot's reaction speed, high-precision trajectory tracking, and full-body coordination, making it an ideal scenario for verifying the control capabilities of such complex robots. Existing methods often suffer from low sample efficiency, difficulty in training convergence, and difficulty in generating smooth movements that conform to biological characteristics when handling such high-dimensional control tasks. Therefore, how to design a method that can meet the high dynamic requirements of table tennis rallies while effectively solving the problem of redundant control of high-dimensional muscles is a pressing technical issue that needs to be addressed. Summary of the Invention

[0005] The present invention aims to solve the problems of difficult control, low training efficiency, and difficulty in completing precise dynamic tasks in the aforementioned high-dimensional musculoskeletal system.

[0006] Therefore, the present invention provides a musculoskeletal robot control method for table tennis match scenarios, including:

[0007] The first aspect of this invention provides a musculoskeletal robot control method for table tennis match scenarios, comprising:

[0008] At the beginning of each hitting cycle, the trajectory of the ping-pong ball is predicted using a trajectory planner based on a physics model. Based on the preset hitting plane and target landing point of the musculoskeletal robot, the expected hitting position, expected hitting time, expected hitting speed and expected hitting direction of the racket held by the musculoskeletal robot are calculated to form high-level control commands.

[0009] The high-level control commands and the current environmental observation status are input into a pre-trained reinforcement learning policy network, which outputs the current target joint position of the musculoskeletal robot.

[0010] An adaptive controller based on forward kinematics is used to map the current target joint position to the activation signals of each muscle, thereby driving the musculoskeletal robot to perform actions.

[0011] In some embodiments, the physics-based trajectory planner uses a hybrid dynamics model to predict the trajectory of the ball, the hybrid dynamics model comprising:

[0012]

[0013] in, The speed at which the ball travels through the air. for The first derivative, It is the acceleration due to gravity. This refers to the air drag coefficient; and These are the ball's incident velocity and exit velocity, respectively, in the direction perpendicular to the plane of the table. The collision recovery coefficient is... ;

[0014] The trajectory planner determines the desired hitting time and the desired hitting position by integrating the hybrid dynamics model until the ball's trajectory intersects with a preset hitting plane.

[0015] In some embodiments, the desired hitting speed of the racket is set to... According to the preset target landing point and expected hitting position The calculation shows that:

[0016]

[0017] in, The time it takes for a musculoskeletal robot to hit a ball and for the ball to land on the opposite side of the table. This is the acceleration due to gravity.

[0018] In some embodiments, the predicted incoming ball velocity, the desired hitting velocity, the contact velocity between the racket and the incoming ball, and the desired hitting direction are respectively... , , and It satisfies the following relationship:

[0019]

[0020]

[0021]

[0022] Among them, the velocity of the incoming ball , is the velocity of the ball when it reaches the preset striking plane, calculated based on the predicted trajectory of the ping-pong ball; racket recovery coefficient. , For desired ball speed The direction.

[0023] In some embodiments, the reinforcement learning policy network employs an asymmetric actor-critic network; wherein the input to the actor network includes current environmental observations. In conjunction with the high-level control commands, the actor network outputs the current target joint position of the musculoskeletal robot. Subscript This represents the time step number within a striking week; the input to the critics' network includes the aforementioned environmental observations. The aforementioned high-level control commands and privileged information;

[0024] Current environmental observation This includes the current proprioception of the musculoskeletal robot and the current position and velocity of the ball in space;

[0025] The privileged information includes at least the remaining time of the current hitting cycle and the racket quality.

[0026] In some embodiments, the training reward function of the reinforcement learning policy network Defined as: ,in, Instruction tracking rewards are provided to encourage the racket to track the higher-level control instructions. A reward is given for a successful landing point; when the ball successfully lands on or near the target landing point on the opponent's side. For gripping rewards, used to encourage the musculoskeletal robot's fingers to grip the racket handle; , , They are respectively with , , The corresponding weighting coefficients.

[0027] In some embodiments, the step of mapping the target joint position to activation signals of each muscle using an adaptive controller based on forward kinematics includes:

[0028] Using the forward kinematics model of a musculoskeletal robot, the expected muscle length of each muscle is calculated based on the target joint position. ;

[0029] For each muscle, the PD controller is used to determine the current muscle length. Current muscle contraction speed and the expected length of the muscle Calculate expected muscle strength :

[0030]

[0031] in, This is a truncation function. The first term is the value before truncation, the second term is the minimum value after truncation, and the third term is the maximum value after truncation. This represents the maximum active muscle force for musculoskeletal robots. and These are the proportional gain and derivative gain of the PD controller, respectively. The range of muscle length;

[0032] Based on the Hill model of muscle dynamics, the desired muscle force Converted to muscle activation :

[0033]

[0034] in, For normalized expected muscle strength, , , These represent the force-length relationship, force-velocity relationship, and passive force function of the musculoskeletal robot, respectively, and are set according to the fitting formula of the actual muscle changes measured in biology. V is the contraction velocity of the muscle at the current moment.

[0035] The second aspect of this invention provides a musculoskeletal robot control device for table tennis matches, comprising:

[0036] The trajectory planning module is configured to predict the trajectory of the table tennis ball based on the physical model at the beginning of each hitting cycle, and calculate the expected hitting position, hitting time, hitting speed and hitting direction of the racket held by the musculoskeletal robot according to the preset hitting plane and target landing point of the musculoskeletal robot, thus forming high-level control commands.

[0037] The decision module is configured to input the high-level control commands and the current environmental observation status into a pre-trained reinforcement learning policy network and output the current target joint position of the musculoskeletal robot.

[0038] An adaptive muscle control module is configured to use a forward kinematics-based adaptive controller to map the current target joint position to activation signals of each muscle, thereby driving the musculoskeletal robot to perform actions.

[0039] A third aspect of the present invention provides an electronic device comprising:

[0040] At least one processor, and a memory communicatively connected to said at least one processor;

[0041] The memory stores instructions executable by the at least one processor, the instructions being configured to perform the musculoskeletal robot control method according to any embodiment of the first aspect of the present invention.

[0042] The fourth aspect of the present invention provides a computer-readable storage medium storing computer instructions for causing the computer to execute the musculoskeletal robot control method according to any embodiment of the first aspect of the present invention.

[0043] The present invention has the following beneficial effects:

[0044] Solved the high-dimensional control problem: By decomposing the control problem into task-level planning, joint-level strategy and muscle-level execution, it avoids blindly searching in the huge action space and achieves effective control of a system with 273 degrees of freedom.

[0045] Improved sample efficiency and stability: The introduction of a physics-based planner as a guide enables reinforcement learning strategies to converge faster; the use of joint position as an intermediate variable is more stable than directly learning muscle activation.

[0046] High-performance rallies were achieved: Experiments show that the method can achieve a very high success rate in returning the ball in a simulated environment, while maintaining a low cost of muscle activation. Attached Figure Description

[0047] Figure 1 This is an overall flowchart of a musculoskeletal robot control method for table tennis match scenarios provided by an embodiment of the present invention.

[0048] Figure 2 This is a schematic diagram of the structure of an electronic device provided in a third aspect embodiment of the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this application clearer, the application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for explaining this application and are not intended to limit this application.

[0050] Conversely, this application covers any alternatives, modifications, equivalent methods, and schemes made within the spirit and scope of this application as defined by the claims. Furthermore, to provide the public with a better understanding of this application, certain specific details are described in detail below. However, this application can be fully understood by those skilled in the art even without these detailed descriptions.

[0051] See Figure 1 The first aspect of this invention provides a musculoskeletal robot control method for table tennis match scenarios, comprising the following steps:

[0052] Step S1: Model-based trajectory planning and instruction generation:

[0053] A hitting cycle is defined as the ball leaving the opponent's racket and flying towards the player's (i.e., the musculoskeletal robot) table, until the ball is hit by the player's racket and leaves the racket. Each hitting cycle has multiple time steps, and in this embodiment of the invention, the step size of the time step is 0.01s. At the beginning of each hitting cycle, a trajectory planner based on a physical model predicts the trajectory of the ping-pong ball and calculates the desired hitting position, desired hitting time, desired hitting speed, and desired hitting direction of the racket held by the robot based on the preset hitting plane and target landing point of the musculoskeletal robot, thus forming high-level control commands.

[0054] Step S2: Generation of target joint positions based on reinforcement learning:

[0055] The high-level control commands generated in step S1 and the current environmental observation state are input into the pre-trained reinforcement learning policy network to output the current target joint position of the musculoskeletal robot.

[0056] Step S3: Kinematic-based adaptive muscle control:

[0057] Using an adaptive controller based on forward kinematics, the target joint position output in step S2 is mapped to the activation signals of each muscle, driving the musculoskeletal robot to perform actions.

[0058] In some embodiments, the trajectory planner in step S1 is triggered once at the beginning of each hitting cycle, and the trajectory planner utilizes the known initial position of the incoming ball. and initial velocity The trajectory of a ping-pong ball is predicted using an integral method based on a hybrid dynamics model. The hybrid dynamics model is as follows:

[0059] During the flight phase: ,in, For the ball's flight speed, Let the first derivative of the ball's flight velocity be... Let be the norm of the ball's flight velocity. The air drag coefficient, It is the acceleration due to gravity;

[0060] During the collision phase: ,in, and These are the ball's incident velocity and exit velocity, respectively, in the direction perpendicular to the plane of the table. The collision recovery coefficient is... The greater the elasticity of the ball and the table, The larger the value, the more elastic the collision between the ball and the table in ideal conditions, causing the ball to bounce back at its original speed. .

[0061] The trajectory planner integrates the aforementioned hybrid dynamics model until the trajectory of the incoming ball intersects with a preset striking plane (in one specific embodiment of this application, the striking plane is defined as an infinitely large plane parallel to the net plane, 1.8 meters away from the net plane, and located on one side of the player's table), thereby determining the expected striking time of the musculoskeletal robot. and expected hitting position .

[0062] In order to return the ping-pong ball to a preset target landing point (in one specific embodiment of this application, the target landing point is the center of the opponent's table), The trajectory planner determines the target landing point. and expected hitting position Calculate the launch velocity required for a musculoskeletal robot to launch the ball. And further calculate the expected hitting speed. The expected hitting position will be calculated based on the expected hitting direction and the desired hitting orientation. Expected time to strike Expected hitting speed And the desired shot direction, forming high-level control instructions. Among them, the launch velocity The following formula is used to calculate:

[0063]

[0064] in, The time it takes for the ball to travel from when the robot hits it until it lands on the opposite side of the table.

[0065] Expected hitting speed In order to achieve the desired launch velocity The contact speed between the robot racket and the incoming ball, combined with the predicted ball incident velocity. and racket recovery coefficient Calculated :

[0066]

[0067] Among them, racket recovery coefficient The greater the elasticity of the volleyball, The larger the ball, the more ideally it would cause a perfectly elastic collision between the ball and the racket, with the ball rebounding at its original speed. ; For desired ball speed The direction;

[0068] Assuming the racket face and the desired hitting speed are at the moment of impact... direction Vertical, therefore the expected direction of the shot (using the normal vector of the racket face). (Indicates) and expected hitting speed The directions are the same, that is: .

[0069] It is understood that step S1 of this embodiment of the invention uses Newton's laws and aerodynamic models to predict the trajectory of the incoming ball and calculates the desired hitting position and desired hitting time on the preset hitting plane. Simultaneously, based on the preset target landing point on the opponent's side, the required hitting speed and orientation of the racket are deduced. The instructions generated in this stage remain constant throughout the entire hitting cycle, providing stable target guidance for the underlying strategy.

[0070] In some embodiments, in step S2, the reinforcement learning policy network is trained using the Proximal Policy Optimization (PPO) algorithm and employs an asymmetric Actor-Critic architecture. The input to the Critic network includes privileged information in addition to the input to the Actor network. This privileged information includes at least: the remaining time of the current hitting cycle and the mass (in kg) of the racket held by the robot. The Critic's output is the state value at future moments, defined as the total reward that can be obtained if the current Actor policy continues to be implemented. The Actor's input includes current environmental observations. and the high-level control commands generated in step S1 (Including expected hitting position, expected hitting time, expected hitting speed, and expected hitting direction), sub-label Representing the time step number within a striking cycle, the Actor's output is the current target joint position of the musculoskeletal robot. Furthermore, the environmental observations received by the Actor at each time step during the striking cycle... This includes the robot's current proprioceptive observations (including joint positions, joint velocities, muscle lengths, muscle contraction speeds, and muscle activation states) and the ball's current position and velocity in space. Compared to directly outputting muscle activation, outputting joint positions leverages the fact that the upper limbs of musculoskeletal robots only have 32 degrees of freedom, significantly reducing dimensionality. A reward function for training the reinforcement learning policy network is designed. This is a composite reward function that includes instruction tracking reward (encouraging the robot to move to the planned position), landing reward (encouraging the ball to land within the opponent's bounds), and grip reward (preventing the racket from falling). Specifically, the reward function... The expression is as follows:

[0071]

[0072] in, For instruction tracking rewards, high-level control instructions are used to encourage racket tracking planning. A high score is awarded when the ball successfully lands at or near the target landing point on the opponent's side of the table. For gripping rewards, used to encourage the musculoskeletal robot's fingers to grip the racket handle; , , They are respectively with , , The corresponding weighting coefficients.

[0073] It is understood that step S2 in this embodiment of the invention adopts an asymmetric actor-critic structure. The actor network receives environmental observations and high-level control instructions from the trajectory planner, and outputs low-dimensional target joint positions (rather than directly outputting muscle activation), reducing the learning difficulty of the policy. The criterion network utilizes privileged information (such as remaining time and racket quality) to assist training, improving the robustness and long-term benefits of the policy.

[0074] In some embodiments, the adaptive controller in step S3 is responsible for achieving the target joint position, including the following steps:

[0075] Step S31: Using the forward kinematics model of the musculoskeletal robot, convert the target joint position output by the reinforcement learning policy network into the target length of each muscle. ;

[0076] Step S32: For each muscle, use the PD controller to determine the current muscle length. Current muscle contraction speed and expected muscle length Calculate expected muscle strength :

[0077]

[0078] in, This is a truncation function. The first term is the value before truncation, the second term is the minimum value after truncation, and the third term is the maximum value after truncation. The maximum active muscle force for a musculoskeletal robot; and These are the proportional gain and derivative gain of the PD controller, respectively. This refers to the length range of muscles, and the length range of different muscles. These should be set separately; the formula generates tension based on the error between the current muscle length and the target length.

[0079] Step S33: Based on the Hill model of muscle dynamics, determine the desired muscle force. Converted into normalized muscle activation signals The virtual muscle groups are input into the robot. The calculation formula is as follows:

[0080]

[0081] in, For normalized expected muscle strength, , , These are the force-length relationship, the force-velocity relationship, and the passive force function, respectively. The specific functional relationships are modeled according to the Hill muscle model in biology, where V is the contraction velocity of the muscle at the current moment.

[0082] It is understood that step S3 of this embodiment of the invention utilizes prior knowledge of biological motion control, which can significantly improve the stability and realism of motion generation.

[0083] Validation of the embodiments of the present invention:

[0084] This method was validated in the MuJoCo physics simulation environment. The task difficulty was gradually increased using a course-based learning approach. Optimal results were achieved after approximately 1-2 billion training steps. Experimental results show that in the MyoChallenge 2025 table tennis track, the method achieved a 100% success rate in the first stage with a muscle activation cost of only 0.02441; in the second stage, it achieved a 64% success rate, significantly outperforming other comparative methods. This demonstrates the effectiveness of hierarchical architecture in handling complex musculoskeletal control tasks.

[0085] A second aspect of the present invention provides a musculoskeletal robot control device for table tennis matches, comprising:

[0086] The trajectory planning module is configured to predict the trajectory of the table tennis ball based on the physical model at the beginning of each hitting cycle, and calculate the expected hitting position, hitting time, hitting speed and hitting direction of the racket held by the robot according to the preset hitting plane and target landing point of the musculoskeletal robot, thus forming high-level control commands.

[0087] The decision-making module is configured to input the aforementioned high-level control commands and the current environmental observation status into a pre-trained reinforcement learning policy network, and output the current target joint position of the musculoskeletal robot.

[0088] The adaptive muscle control module is configured to use a forward kinematics-based adaptive controller to map the target joint positions to activation signals of each muscle, thereby driving the musculoskeletal robot to perform actions.

[0089] It should be noted that the foregoing explanation of the embodiments of the musculoskeletal robot control method also applies to the musculoskeletal robot control device of this embodiment, and will not be repeated here.

[0090] To implement the above embodiments, this invention also proposes a computer-readable storage medium storing a computer program thereon, which is executed by a processor to perform the musculoskeletal robot control method of the above embodiments.

[0091] The following is for reference. Figure 2 The diagram illustrates a structural schematic of an electronic device suitable for implementing embodiments of the present invention. It should be noted that the electronic device in the embodiments of the present invention may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs, desktop computers, and servers. Figure 2 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0092] like Figure 2As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 101, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 102 or a program loaded from a storage device 108 into a random access memory (RAM) 103. The RAM 103 also stores various programs and data required for the operation of the electronic device. The processing unit 101, ROM 102, and RAM 103 are interconnected via a bus 104. An input / output (I / O) interface 105 is also connected to the bus 104.

[0093] Typically, the following devices can be connected to I / O interface 105: input devices 106 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, etc.; output devices 107 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 108 including, for example, magnetic tapes, hard disks, etc.; and communication devices 109. Communication device 109 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 2 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0094] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, this embodiment includes a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network via communication device 109, or installed from storage device 108, or installed from ROM 102. When the computer program is executed by processing device 101, it performs the functions defined above in the methods of embodiments of this disclosure.

[0095] It should be noted that the computer-readable medium described above in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0096] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0097] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the aforementioned musculoskeletal robot control method.

[0098] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and Python, as well as conventional procedural programming languages ​​such as the "C-" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0099] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0100] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0101] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as will be understood by those skilled in the art to which embodiments of this application pertain.

[0102] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0103] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0104] Those skilled in the art will understand that implementing all or part of the steps of the methods in the above embodiments can be accomplished by instructing related hardware through a program. The developed program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0105] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0106] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A musculoskeletal robot control method for table tennis matches, characterized in that, include: At the beginning of each hitting cycle, the trajectory of the ping-pong ball is predicted by a trajectory planner based on a physics model. Based on the preset hitting plane and target landing point of the musculoskeletal robot, the expected hitting position, expected hitting time, expected hitting speed and expected hitting direction of the racket held by the musculoskeletal robot are calculated to form high-level control commands. The high-level control commands and the current environmental observation status are input into a pre-trained reinforcement learning policy network, which outputs the current target joint position of the musculoskeletal robot. An adaptive controller based on forward kinematics is used to map the current target joint position to the activation signals of each muscle, thereby driving the musculoskeletal robot to perform actions. The trajectory planner based on the physics model uses a hybrid dynamics model to predict the trajectory of the ball, the hybrid dynamics model including: in, The speed at which the ball travels through the air. for The first derivative, It is the acceleration due to gravity. This refers to the air drag coefficient; and These are the ball's incident velocity and exit velocity, respectively, in the direction perpendicular to the plane of the table. The collision recovery coefficient is... ; The trajectory planner determines the desired hitting time and the desired hitting position by integrating the hybrid dynamics model until the ball's trajectory intersects with a preset hitting plane. Let the desired hitting speed of the racket be... According to the preset target landing point and expected hitting position The calculation shows that: in, The time it takes for a musculoskeletal robot to hit a ball and for the ball to land on the opposite side of the table. It is the acceleration due to gravity; Let the predicted incoming ball velocity, the desired hitting velocity, the contact velocity between the racket and the incoming ball, and the desired hitting direction be respectively... , , and It satisfies the following relationship: Among them, the velocity of the incoming ball , is the velocity of the ball when it reaches the preset striking plane, calculated based on the predicted trajectory of the ping-pong ball; racket recovery coefficient. , For desired ball speed The direction.

2. The musculoskeletal robot control method according to claim 1, characterized in that, The reinforcement learning policy network employs an asymmetric actor-critic network; wherein, the input to the actor network includes current environmental observations. In conjunction with the high-level control commands, the actor network outputs the current target joint position of the musculoskeletal robot. Subscript This represents the time step number within a striking cycle; the input to the critic network includes the aforementioned environmental observations. The aforementioned high-level control commands and privileged information; Current environmental observation This includes the current proprioception of the musculoskeletal robot and the current position and velocity of the ball in space; The privileged information includes at least the remaining time of the current hitting cycle and the racket quality.

3. The musculoskeletal robot control method according to claim 1, characterized in that, The training reward function of the reinforcement learning policy network Defined as: ,in, Instruction tracking rewards are provided to encourage the racket to track the higher-level control instructions. A reward is given for a successful landing point; when the ball successfully lands on or near the target landing point on the opponent's side. For gripping rewards, used to encourage the musculoskeletal robot's fingers to grip the racket handle; , , They are respectively with , , The corresponding weighting coefficients.

4. The musculoskeletal robot control method according to claim 1, characterized in that, The method of using an adaptive controller based on forward kinematics to map the target joint position to activation signals of each muscle includes: Using the forward kinematics model of a musculoskeletal robot, the expected muscle length of each muscle is calculated based on the target joint position. ; For each muscle, the PD controller is used to determine the current muscle length. Current muscle contraction speed and the expected length of the muscle Calculate expected muscle strength : in, This is a truncation function. The first term is the value before truncation, the second term is the minimum value after truncation, and the third term is the maximum value after truncation. This represents the maximum active muscle force for musculoskeletal robots. and These are the proportional gain and derivative gain of the PD controller, respectively. The range of muscle length; Based on the Hill model of muscle dynamics, the desired muscle force Converted to muscle activation : in, For normalized expected muscle strength, , , These are the force-length relationship, force-velocity relationship, and passive force function of the musculoskeletal robot, respectively, set according to the fitting formula of the actual muscle changes measured in biology.

5. A musculoskeletal robot control device for table tennis matches, characterized in that, include: The trajectory planning module is configured to predict the trajectory of the table tennis ball based on the physical model at the beginning of each hitting cycle, and calculate the expected hitting position, expected hitting time, expected hitting speed and expected hitting direction of the racket held by the musculoskeletal robot according to the preset hitting plane and target landing point of the musculoskeletal robot, and form high-level control commands. The decision module is configured to input the high-level control commands and the current environmental observation status into a pre-trained reinforcement learning policy network and output the current target joint position of the musculoskeletal robot. An adaptive muscle control module is configured to use a forward kinematics-based adaptive controller to map the current target joint position to activation signals of each muscle, thereby driving the musculoskeletal robot to perform actions. The trajectory planner based on the physics model uses a hybrid dynamics model to predict the trajectory of the ball, the hybrid dynamics model including: in, The speed at which the ball travels through the air. for The first derivative, It is the acceleration due to gravity. This refers to the air drag coefficient; and These are the ball's incident velocity and exit velocity, respectively, in the direction perpendicular to the plane of the table. The collision recovery coefficient is... ; The trajectory planner determines the desired hitting time and the desired hitting position by integrating the hybrid dynamics model until the ball's trajectory intersects with a preset hitting plane. Let the desired hitting speed of the racket be... According to the preset target landing point and expected hitting position The calculation shows that: in, The time it takes for a musculoskeletal robot to hit a ball and for the ball to land on the opposite side of the table. It is the acceleration due to gravity; Let the predicted incoming ball velocity, the desired hitting velocity, the contact velocity between the racket and the incoming ball, and the desired hitting direction be respectively... , , and It satisfies the following relationship: Among them, the velocity of the incoming ball , is the velocity of the ball when it reaches the preset striking plane, calculated based on the predicted trajectory of the ping-pong ball; racket recovery coefficient. , For desired ball speed The direction.

6. An electronic device, characterized in that, include: At least one processor, and a memory communicatively connected to said at least one processor; The memory stores instructions that can be executed by the at least one processor, the instructions being configured to perform the musculoskeletal robot control method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the musculoskeletal robot control method according to any one of claims 1 to 4.

Images