A robot motion control method, device, equipment and readable storage medium

Abstract

The present invention discloses a robot motion control method, device, equipment and readable storage medium, which are applied to the field of intelligent control, including: obtaining the current robot state; based on the current robot state, using the motion control model, obtaining the control instruction at the next moment, which is used to control the robot; when training the motion control model, updating parameters according to the state prediction reward, the state prediction reward is determined according to the robot predicted state and the robot actual state, and the robot predicted state is determined based on the training hidden variables; the hidden variables are unobservable random variables, including parameters that can predict the future state of the robot. This application makes the robot motion control model's prediction of the future robot state closer to the robot's actual state through the state prediction reward, so that the control instruction generated at the next moment is more suitable for the robot's actual state, thereby improving the stability of subsequent robot control.

Description

Technical Field

[0001] The present invention relates to the field of intelligent control technology, and in particular to a robot motion control method, device, equipment and readable storage medium. Background Art

[0002] At present, the motion control of robots is mainly based on the dynamic model method. This method is based on the robot model and the environment model for calculation. In addition, the underlying high-frequency control makes the execution of actions in a certain environment relatively stable and accurate, but it cannot cope well with changes in the robot model and the environment model. The reinforcement learning model method can make the generated motion control strategy able to cope with model changes by randomizing the robot model and the environment model during training. However, the control method based on reinforcement learning directly generates the underlying joint control instructions at the current moment, and the generated actions are often more random.

[0003] Therefore, the current method for controlling the movement of the robot has the technical problem that the robot movement control is not stable enough. Summary of the invention

[0004] In view of this, an object of the present invention is to provide a robot motion control method, which solves the technical problem that the robot motion control is not stable enough in the prior art.

[0005] In order to solve the above technical problems, the present invention provides a robot motion control method, comprising:

[0006] Get the current robot status;

[0007] Based on the current robot state, using the motion control model, a control instruction at the next moment is obtained to control the robot;

[0008] Among them, when the motion control model is trained, parameters are updated according to the state prediction reward, and the state prediction reward is determined according to the robot's predicted state and the robot's actual state. The robot's predicted state is determined based on training latent variables; the latent variables are unobservable random variables, including parameters that can predict the robot's future state.

[0009] Optionally, based on the current robot state, a motion control model is used to obtain hidden variables, and the obtained hidden variables do not participate in the control of the robot.

[0010] Optionally, the hidden variables include at least one of environmental characteristics, historical behavior patterns of the robot, and load capacity of the robot.

[0011] Optionally, the next moment is related to a parameter update period during training of the motion control model.

[0012] Optionally, the training process of the motion control model includes:

[0013] Based on the training robot state at the current moment, the training control instructions and training hidden variables at the next moment are obtained by using the motion control model to be trained;

[0014] Based on the training latent variables, using the to-be-trained motion control model to make predictions, to obtain the predicted state of the robot at the next moment;

[0015] Utilizing a simulation engine to execute the training control instructions to obtain the actual state of the robot;

[0016] Determining a state prediction reward according to the predicted state of the robot and the actual state of the robot;

[0017] According to the state prediction reward, the parameters of the motion control model to be trained are updated to obtain the motion control model.

[0018] Optionally, the using of the simulation engine to execute the training control instruction also obtains the environment state;

[0019] The training process of the motion control model also includes:

[0020] Determining a task reward according to the actual state of the robot and the state of the environment;

[0021] According to the state prediction reward, updating the parameters of the motion control model to be trained includes:

[0022] The state prediction reward and the task reward are summed as a training reward;

[0023] According to the training reward, the parameters of the motion control model to be trained are updated.

[0024] Optionally, the state prediction reward is j; wherein, , is the actual state of the robot, A state is predicted for the robot.

[0025] The present application also provides a robot motion control device, comprising:

[0026] The acquisition module is used to obtain the current robot status;

[0027] A control module, used to obtain a control instruction at the next moment based on the current robot state and using a motion control model, so as to control the robot;

[0028] Among them, when the motion control model is trained, parameters are updated according to the state prediction reward, and the state prediction reward is determined according to the robot's predicted state and the robot's actual state. The robot's predicted state is determined based on training latent variables; the latent variables are unobservable random variables, including parameters that can predict the robot's future state.

[0029] The present application also provides a robot motion control device, comprising:

[0030] Memory for storing computer programs;

[0031] A processor is used to execute the computer program to implement the steps of the robot motion control method as described above.

[0032] The present application also provides a readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the robot motion control method as described above are implemented.

[0033] It can be seen that the beneficial effects of the present invention are: when the motion control model in the present application is trained, the predicted state of the robot is determined based on hidden variables, the state prediction reward is determined according to the predicted state of the robot and the actual state of the robot, and then the model parameters are updated according to the state prediction reward; the hidden variables are unobservable random variables, including parameters that can predict the future state of the robot, so that the robot motion control model's prediction of the future robot state is closer to the actual state of the robot through the state prediction reward, so that the control instructions generated at the next moment are more suitable for the actual state of the robot, thereby improving the stability of subsequent control of the robot.

[0034] In addition, the present invention also provides a robot motion control device, equipment and readable storage medium, which also have the above-mentioned beneficial effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0036] Figure 1 A flowchart of a robot motion control method provided by an embodiment of the present invention;

[0037] Figure 2 A flowchart of another robot motion control method provided by an embodiment of the present invention;

[0038] Figure 3A schematic diagram of a robot motion control method provided by an embodiment of the present invention;

[0039] Figure 4 A flowchart of a robot motion control model training method provided by an embodiment of the present invention;

[0040] Figure 5 A flowchart of another robot motion control model training method provided by an embodiment of the present invention;

[0041] Figure 6 A schematic diagram of a robot motion control model training method provided by an embodiment of the present invention;

[0042] Figure 7 A schematic diagram of the structure of a robot motion control device provided by an embodiment of the present invention;

[0043] Figure 8 A schematic structural diagram of a robot motion control device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] Please refer to Figure 1 , Figure 1 A flowchart of a robot motion control method provided by an embodiment of the present invention. The method may include:

[0046] S101, obtaining the current robot status.

[0047] The execution subject of this embodiment is an electronic device. For example, the electronic device in this embodiment can be a computer or the like.

[0048] The robot state in this embodiment may include at least one of the position of each joint of the robot, the speed of each joint of the robot, and the feedback data of the robot inertial sensor, and the feedback data of the robot inertial sensor includes parameters such as acceleration, angular velocity, and posture. It can be understood that by providing the state related to the joints related to the robot, the subsequent accurate control of the robot movement can be achieved.

[0049] S102, based on the current robot state, using the motion control model, obtain the control instructions at the next moment, which are used to control the robot; wherein, during the motion control model training, parameters are updated according to the state prediction reward, the state prediction reward is determined according to the robot's predicted state and the robot's actual state, and the robot's predicted state is determined based on the training latent variables; the latent variables are unobservable random variables, including parameters that can predict the robot's future state.

[0050] This embodiment does not limit the specific type of the motion control model. For example, the motion control model in this embodiment can be a reinforcement learning model, such as Q-Learning (a reinforcement learning method based on value function, which learns the expected utility of taking a specific action in a given state, i.e., Q value), SARSA (an algorithm similar to Q-Learning, except that it uses the actual next state and reward to update the current Q value instead of using the method of maximizing future rewards). Since the reinforcement learning model has strong adaptability and flexibility, the adaptability and flexibility of the motion control model are improved.

[0051] In this embodiment, the control instruction at the next moment is obtained according to the motion control model, and the control instruction is the expected position instruction for each joint of the robot. The next moment is related to the parameter update cycle during the motion control model training. It can be understood that the parameter update cycle is , then the next moment is , t is the current time.

[0052] It can be understood that based on the current robot state, using the motion control model, hidden variables are also obtained, and the obtained hidden variables do not participate in the control of the robot.

[0053] It should be further explained that, in order to improve the accuracy of the control instructions generated by the motion control model at the next moment, the above-mentioned hidden variables may include at least one of the environmental characteristics, the robot's historical behavior pattern and the robot's load capacity. It is understandable that the hidden variables in this embodiment may include the environment in which the robot is located; or the hidden variables in this embodiment may also include the robot's historical operating state; or the hidden variables in this embodiment may also include the maximum load mass that the robot can handle when performing a task.

[0054] Understandably, latent variables are random variables that cannot be directly observed but play an important role in the model's predictions and decisions. For robot behavior, these latent variables can include a variety of factors, such as environmental characteristics, the robot's historical behavior patterns, and the robot's physical and operational properties, such as payload capacity. By taking these factors into account, the model can better understand and predict the robot's behavior and performance in a specific environment.

[0055] Since the motion control model determines the state prediction reward based on hidden variables during training and updates parameters according to the state prediction reward, the state prediction reward makes the robot motion control model's prediction of the future robot state closer to the robot's actual state, making the control instructions generated at the next moment more suitable for the robot's actual state, thereby improving the stability of subsequent robot control.

[0056] To make the present invention easier to understand, please refer to Figure 2 , Figure 2 A flowchart of another robot motion control method provided by an embodiment of the present invention. The method may include:

[0057] S201, obtaining the current robot status.

[0058] Please refer to Figure 3 , Figure 3 A schematic diagram of a robot motion control method provided by an embodiment of the present invention.

[0059] S202, based on the current robot state, using the motion control model, obtain the control instructions and hidden variables at the next moment, the control instructions at the next moment are used to control the robot; the hidden variables do not participate in the control of the robot.

[0060] After the robot executes the control instruction at the next moment, the latest robot state can be obtained. The motion control model regenerates the latest control instruction at the next moment based on the latest robot state to realize the control of the robot.

[0061] When the motion control model in this embodiment is trained, the robot's predicted state is determined based on hidden variables, the state prediction reward is determined based on the robot's predicted state and the robot's actual state, and then the model parameters are updated based on the state prediction reward. The hidden variables in this embodiment include predictions of future states. Hidden variables can represent factors that are not directly observed by the robot, such as the potential state of the environment. By introducing these variables, the model can express problems more richly, thereby having a deeper understanding of complex scenarios. By abstracting some unobservable factors, the model is not only limited to specific observation data, but can also make inferences and decisions. This generalization ability is the key to dealing with unknown environments.

[0062] Please refer to Figure 4 , Figure 4 A flowchart of a robot motion control model training method provided by an embodiment of the present invention. The method may include:

[0063] S301, based on the training robot state at the current moment, using the motion control model to be trained, obtain the training control instructions and training hidden variables at the next moment.

[0064] The execution subject of this embodiment is an electronic device. For example, the electronic device in this embodiment can be a computer or the like.

[0065] This embodiment does not limit the specific type of the motion control model to be trained. For example, the motion control model to be trained in this embodiment may be a reinforcement learning model, and the reinforcement learning model may include a policy network and a decoding network.

[0066] It should be further explained that the above-mentioned training control instructions and training hidden variables at the next moment are obtained based on the training robot state at the current moment by using the motion control model to be trained, which may include:

[0067] Based on the training robot state at the current moment, the policy network of the motion control model to be trained is used to obtain the training control instructions and training hidden variables at the next moment.

[0068] Correspondingly, when the robot's motion control is subsequently performed based on this model, the strategy network of the motion control model is also used to obtain control instructions.

[0069] S302, based on the training latent variables, using the motion control model to be trained to make predictions to obtain the predicted state of the robot at the next moment.

[0070] It should be further explained that the above-mentioned prediction based on the training latent variables and using the motion control model to be trained to obtain the predicted state of the robot at the next moment may include:

[0071] Based on the training latent variables, the decoding network of the motion control model to be trained is used to make predictions to obtain the predicted state of the robot at the next moment.

[0072] S303, using the simulation engine to execute the training control instructions to obtain the actual state of the robot.

[0073] This embodiment does not limit the order of S302 and S303. For example, S302 may be executed first and then S303; or this embodiment may execute S303 first and then S302; or this embodiment may execute S302 and S303 simultaneously. The actual state of the robot obtained by the simulation engine may be used as the training robot state.

[0074] S304, determining a state prediction reward according to the predicted state of the robot and the actual state of the robot.

[0075] This embodiment can determine the accuracy of the prediction by comparing the predicted state of the robot with the actual state of the robot, and obtain a state prediction reward; wherein, the closer the predicted state of the robot is to the actual state of the robot, the closer the state prediction reward is to 1, otherwise the state prediction reward is closer to 0.

[0076] S305, updating the parameters of the motion control model to be trained according to the state prediction reward to obtain the motion control model.

[0077] This embodiment does not limit the specific method of updating the model parameters according to the state prediction reward. The task reward can be determined, and then the parameters of the motion control model to be trained are updated according to the state prediction reward and the task reward.

[0078] It should be further explained that the above-mentioned updating the parameters of the motion control model to be trained according to the state prediction reward to obtain the motion control model may include:

[0079] According to the state prediction reward, the parameters of the policy network and the decoding network of the motion control model to be trained are updated to obtain the motion control model.

[0080] By adopting the robot motion control model training method provided by the embodiment of the present invention, the motion control model outputs the control instructions and hidden variables at the next moment during the training process, predicts the future robot state based on the hidden variables, and updates the model parameters according to the predicted robot state and the actual robot state, so that the model's prediction of the future robot state is closer to the actual robot state, and the subsequent control instructions generated based on the model are more accurate, thereby improving the stability of subsequent robot control.

[0081] In order to make the present invention easier to understand, please refer to Figure 5 , Figure 5 A flowchart of another robot motion control model training method provided by an embodiment of the present invention. The method may include:

[0082] S401, based on the actual state of the current robot obtained by the simulation engine, the policy network of the motion control model to be trained is used to obtain the training control instructions and training hidden variables at the next moment.

[0083] For easier understanding, please refer to Figure 6 , Figure 6 A schematic diagram of a framework of a robot motion control model training method provided by an embodiment of the present invention.

[0084] This embodiment does not limit the specific type of the policy network, which may be a convolutional neural network (CNN) or a recurrent neural network, and the structure may be a multi-layer perceptron network structure (MLP).

[0085] S402, based on the training latent variables, using the decoding network of the motion control model to be trained to make predictions to obtain the predicted state of the robot at the next moment.

[0086] This embodiment does not limit the specific type of the decoding network, which may be a convolutional neural network (CNN) or a recurrent neural network, and the structure may be a multi-layer perceptron network structure (MLP).

[0087] S403, using the simulation engine to execute the training control instructions to obtain the actual state of the robot and the state of the environment.

[0088] S404, determining a state prediction reward according to the predicted state of the robot and the actual state of the robot.

[0089] This embodiment does not limit the specific determination method of the state prediction reward, and the state prediction reward may be j; wherein, , is the actual state of the robot, Predict states for the robot.

[0090] S405, determining the task reward according to the actual state of the robot and the state of the environment.

[0091] This embodiment does not limit the specific determination method of the task reward, and the task reward can be determined according to the actual state of the robot, the state of the environment, and the expected state. The task reward needs to be designed by the user for the training task.

[0092] S406, summing the state prediction reward and the task reward as the training reward.

[0093] S407, updating parameters of the policy network and the decoding network of the motion control model to be trained according to the training reward.

[0094] This embodiment uses the sum of the state prediction reward and the task reward as the total training reward to simultaneously update the policy network and the decoding network. The network update algorithm can adopt the proximal policy update algorithm.

[0095] When performing model reward processing, the embodiment of the present invention uses state prediction rewards and task rewards together to perform model reward processing, thereby improving the generalization ability, stability and reliability of the motion control model.

[0096] The following is an introduction to a robot motion control device provided by an embodiment of the present invention. The robot motion control device described below and the robot motion control method described above can be referenced to each other.

[0097] Please refer to Figure 7 , Figure 7 A schematic diagram of the structure of a robot motion control device provided by an embodiment of the present invention may include:

[0098] The acquisition module 100 is used to obtain the current robot state;

[0099] The control module 200 is used to obtain the control instruction at the next moment based on the current robot state and using the motion control model to control the robot;

[0100] Among them, when the motion control model is trained, parameters are updated according to the state prediction reward, and the state prediction reward is determined according to the robot's predicted state and the robot's actual state. The robot's predicted state is determined based on training latent variables; the latent variables are unobservable random variables, including parameters that can predict the robot's future state.

[0101] Furthermore, based on the above embodiment, based on the current robot state, a motion control model is used to obtain hidden variables, and the obtained hidden variables do not participate in the control of the robot.

[0102] Further, based on any of the above embodiments, the hidden variables include at least one of environmental characteristics, historical behavior patterns of the robot, and load capacity of the robot.

[0103] Further, based on any of the above embodiments, the next moment is related to a parameter update period during training of the motion control model.

[0104] Further, based on any of the above embodiments, the training process of the motion control model includes:

[0105] Based on the training robot state at the current moment, the training control instructions and training hidden variables at the next moment are obtained by using the motion control model to be trained;

[0106] Based on the training latent variables, using the to-be-trained motion control model to make predictions, to obtain the predicted state of the robot at the next moment;

[0107] Utilizing a simulation engine to execute the training control instructions to obtain the actual state of the robot;

[0108] Determining a state prediction reward according to the predicted state of the robot and the actual state of the robot;

[0109] According to the state prediction reward, the parameters of the motion control model to be trained are updated to obtain the motion control model.

[0110] Further, based on any of the above embodiments, the using of the simulation engine to execute the training control instruction also obtains the environment state;

[0111] The training process of the motion control model also includes:

[0112] Determining a task reward according to the actual state of the robot and the state of the environment;

[0113] According to the state prediction reward, updating the parameters of the motion control model to be trained includes:

[0114] The state prediction reward and the task reward are summed as a training reward;

[0115] According to the training reward, the parameters of the motion control model to be trained are updated.

[0116] Further, based on the above embodiment, the state prediction reward is j; wherein, , is the actual state of the robot, A state is predicted for the robot.

[0117] It should be noted that the order of the modules and units in the above robot motion control device can be changed without affecting the logic.

[0118] A robot motion control device provided by an embodiment of the present invention is introduced below. The robot motion control device described below and the robot motion control method described above can refer to each other.

[0119] Please refer to Figure 8 , Figure 8 A schematic diagram of the structure of a robot motion control device provided by an embodiment of the present invention may include:

[0120] A memory 10, used for storing computer programs;

[0121] The processor 20 is used to execute the computer program to implement the above-mentioned robot motion control method.

[0122] The memory 10 , the processor 20 , and the communication interface 30 all communicate with each other via a communication bus 40 .

[0123] In the embodiment of the present invention, the memory 10 is used to store one or more programs, and the program may include program code, and the program code includes computer operation instructions. In the embodiment of the present invention, the memory 10 may store programs for implementing the following functions:

[0124] Get the current robot status;

[0125] Based on the current robot state, the motion control model is used to obtain the control instructions at the next moment to control the robot; wherein, during the motion control model training, parameters are updated according to the state prediction reward, the state prediction reward is determined according to the robot's predicted state and the robot's actual state, and the robot's predicted state is determined based on the training latent variables; latent variables are unobservable random variables, including parameters that can predict the robot's future state.

[0126] In a possible implementation, the memory 10 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function, etc.; the data storage area may store data created during use.

[0127] In addition, the memory 10 may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include an NVRAM. The memory stores an operating system and operating instructions, executable modules or data structures, or a subset thereof, or an extended set thereof, wherein the operating instructions may include various operating instructions for implementing various operations. The operating system may include various system programs for implementing various basic tasks and processing hardware-based tasks.

[0128] The processor 20 may be a central processing unit (CPU), an application specific integrated circuit, a digital signal processor, a field programmable gate array or other programmable logic device, a microprocessor or any conventional processor, etc. The processor 20 may call a program stored in the memory 10 .

[0129] The communication interface 30 may be an interface of a communication module, and is used to connect to other devices or systems.

[0130] Of course, it should be noted that Figure 8 The structure shown does not constitute a limitation on the robot motion control device in the embodiment of the present invention. In actual applications, the robot motion control device may include Figure 8 More or fewer components than shown, or combinations of certain components.

[0131] The following is an introduction to a readable storage medium provided in an embodiment of the present invention. The readable storage medium described below and the robot motion control method described above can be referenced to each other.

[0132] The present invention also provides a readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned robot motion control method are implemented.

[0133] The computer-readable storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0134] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0135] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0136] Finally, it should be noted that, in this article, relationships such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variations are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0137] The above is a detailed introduction to a robot motion control method, device, equipment and readable storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A robot motion control method, characterized in that: include: Get the current robot status; Based on the current robot state, using the motion control model, a control instruction at the next moment is obtained to control the robot; The training process of the motion control model includes: Based on the training robot state at the current moment, the training control instructions and training hidden variables at the next moment are obtained by using the motion control model to be trained; the hidden variables are unobservable random variables; Based on the training latent variables, using the to-be-trained motion control model to make predictions, and obtain the predicted state of the robot at the next moment; Utilizing a simulation engine to execute the training control instructions to obtain the actual state of the robot; Determining a state prediction reward according to the predicted state of the robot and the actual state of the robot; According to the state prediction reward, the parameters of the motion control model to be trained are updated to obtain the motion control model.

2. The robot motion control method according to claim 1, characterized in that: Based on the current robot state, a motion control model is used to obtain hidden variables, which do not participate in the control of the robot.

3. The robot motion control method according to claim 1, characterized in that: The hidden variables include at least one of environmental characteristics, historical behavior patterns of the robot, and the load capacity of the robot.

4. The robot motion control method according to claim 1, characterized in that: The next moment is related to a parameter update period during training of the motion control model.

5. The robot motion control method according to any one of claims 1 to 4, characterized in that: The training control instruction is executed by the simulation engine, and the environment state is obtained; The training process of the motion control model also includes: Determining a task reward according to the actual state of the robot and the state of the environment; According to the state prediction reward, updating the parameters of the motion control model to be trained includes: The state prediction reward and the task reward are summed as a training reward; According to the training reward, the parameters of the motion control model to be trained are updated.

6. The robot motion control method according to claim 1, characterized in that: The state prediction reward is j; where , is the actual state of the robot, A state is predicted for the robot.

7. A robot motion control device, characterized in that: include: The acquisition module is used to obtain the current robot status; A control module, used to obtain a control instruction at the next moment based on the current robot state and using a motion control model, so as to control the robot; The training process of the motion control model includes: Based on the training robot state at the current moment, the training control instructions and training hidden variables at the next moment are obtained by using the motion control model to be trained; the hidden variables are unobservable random variables; Based on the training latent variables, using the to-be-trained motion control model to make predictions, and obtain the predicted state of the robot at the next moment; Utilizing a simulation engine to execute the training control instructions to obtain the actual state of the robot; Determining a state prediction reward according to the predicted state of the robot and the actual state of the robot; According to the state prediction reward, the parameters of the motion control model to be trained are updated to obtain the motion control model.

8. A robot motion control device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the computer program to implement the steps of the robot motion control method according to any one of claims 1 to 6.

9. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the robot motion control method according to any one of claims 1 to 6 are implemented.

Images