Robot control method, device, robot and storage medium

Abstract

The present disclosure provides a robot control method, device, robot and storage medium. The method includes: obtaining the posture information of the robot and the depth image collected by the image acquisition component; inputting the posture information and the depth image into the motion control model to generate the motion command of the robot, and controlling the robot to execute the motion command; wherein the motion control model is trained in the following manner: based on the posture information of the robot in the training environment and the depth image collected by the image acquisition component input into the motion control model to obtain the motion command sample, and the posture information of the simulated robot in the simulation environment and the attribute information of the simulation environment input into the motion control model to obtain the motion command label, the motion control model is adjusted network parameters. The above method can control the robot to complete the movement more accurately, and ensure the safety of the robot during the movement.

Description

Technical Field

[0001] The present disclosure relates to the field of robotics technology, and in particular to a robot control method, device, robot, and storage medium. Background Art

[0002] In recent years, legged robots have made significant progress in autonomous mobility and environmental adaptability. Modern legged robots have achieved efficient crossing over complex terrains, including steps, hurdles, stairs and other obstacles, through continuously optimized control algorithms and advanced sensor technology. In particular, the application of end-to-end deep learning methods and reinforcement learning technology has greatly improved the adaptability and intelligence of robots in unknown environments.

[0003] In the related art, the method of training the model for controlling the robot's motion often relies on pre-labeled model training data. However, the amount of pre-labeled training data is small, which leads to low accuracy of the trained motion control model, and then there will be the problem of low accuracy of the motion instructions generated based on the trained motion control model, which cannot guarantee the accuracy and safety of the robot's motion. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides a robot control method, device, robot and storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, the present application provides a robot control method, the method comprising:

[0006] Obtaining the robot's posture information and the depth image collected by the image acquisition component;

[0007] Inputting the position information and the depth image into a motion control model to generate motion instructions for the robot, and controlling the robot to execute the motion instructions;

[0008] The motion control model is trained in the following way:

[0009] The motion control model is network-adjusted based on motion instruction samples obtained by inputting the motion control model with the posture information of the robot in the training environment and the depth image captured by the image acquisition component, and motion instruction labels obtained by inputting the motion control model with the posture information of the simulated robot in the simulation environment and the attribute information of the simulation environment.

[0010] In combination with any embodiment of the present disclosure, the motion control model is adjusted network parameters based on the motion command samples obtained by inputting the motion control model based on the posture information of the robot in the training environment and the depth image collected by the image acquisition component, and the motion command labels obtained by inputting the motion control model based on the posture information of the simulated robot in the simulation environment and the attribute information of the simulation environment, including:

[0011] Inputting the posture information of the robot in the training environment and the depth image into the motion control model to generate motion instruction samples of the robot;

[0012] Inputting the pose information of the simulated robot in the simulated environment and the attribute information of the simulated environment into the motion control model to generate a motion instruction label of the robot;

[0013] The motion control model is network-adjusted according to the motion command sample and the motion command label.

[0014] In conjunction with any embodiment of the present disclosure, the motion control model is a reinforcement learning model;

[0015] The method further comprises:

[0016] Inputting the position and posture information of the simulated robot in the simulated environment and the attribute information of the simulated environment into the motion control model to generate motion instructions for the simulated robot;

[0017] Controlling the simulated robot to execute the motion instruction and obtaining the motion result of the simulated robot in the simulation environment;

[0018] Determining a model reward value according to the movement instruction and the movement result;

[0019] According to the model reward value, the network parameters of the motion control model are updated until a convergence condition is met.

[0020] In combination with any embodiment of the present disclosure, the method further includes:

[0021] Inputting the pose information of the simulated robot into a state estimation network to generate state information of the simulated robot, wherein the state information is used to characterize attribute information of the simulated environment;

[0022] Determining a network loss of the state estimation network according to the attribute information of the simulation environment and the state information of the simulation robot;

[0023] According to the network loss, the state estimation network is adjusted until a convergence condition is met.

[0024] In combination with any embodiment of the present disclosure, the step of inputting the position information and the depth image into a motion control model to generate a motion instruction for the robot at the next moment includes:

[0025] Inputting the posture information of the robot into the state estimation network to generate state information of the robot;

[0026] The position information, the state information and the depth image are input into a motion control model to generate a motion instruction for the robot at the next moment.

[0027] In combination with any embodiment of the present disclosure, the step of inputting the pose information of the simulated robot in the simulated environment and the attribute information of the simulated environment into the motion control model to generate the motion instruction label of the robot includes:

[0028] Encoding the attribute information of the simulation environment to generate depth attribute information;

[0029] The pose information of the simulated robot in the simulated environment and the depth attribute information are input into the motion control model to generate a motion instruction label for the robot.

[0030] In combination with any embodiment of the present disclosure, the step of inputting the posture information of the robot in the training environment and the depth image into the motion control model to generate motion instruction samples of the robot includes:

[0031] Determining the movement direction and terrain information of the robot according to the depth image;

[0032] The position information, the moving direction and the terrain information of the robot in the training environment are input into the motion control model to generate motion instruction samples of the robot.

[0033] In combination with any embodiment of the present disclosure, the step of inputting the posture information of the robot in the training environment and the depth image into the motion control model to generate motion instruction samples of the robot includes:

[0034] Encoding the depth image to generate depth terrain information;

[0035] The posture information of the robot in the training environment and the depth terrain information are input into the motion control model to generate motion instruction samples of the robot.

[0036] In combination with any embodiment of the present disclosure, the attribute information of the simulated robot includes ground friction information of the simulated environment, sole contact force information of the simulated robot, and movement direction.

[0037] In combination with any embodiment of the present disclosure, during the training process of the motion control model, the simulation environment is obtained by simulating the training environment, and the training environment includes a variety of terrain environments.

[0038] According to a second aspect of the embodiments of the present disclosure, the present application provides a robot control method, the method comprising:

[0039] Obtain the robot's current posture information and the depth image collected by the image acquisition component;

[0040] Inputting the position information and the depth image into a motion control model to generate a motion instruction for the robot, and controlling the robot to execute the motion instruction at the next moment;

[0041] Among them, the motion control model is built based on a deep learning model, and the motion control model is used to control the movement of the robot so that the robot can move in a variety of terrain environments.

[0042] According to a third aspect of the embodiments of the present disclosure, the present application provides a robot control device, the device comprising:

[0043] A first acquisition module is used to acquire the posture information of the robot and the depth image acquired by the image acquisition component;

[0044] An instruction generation module, used for inputting the posture information and the depth image into a motion control model to generate motion instructions for the robot, and controlling the robot to execute the motion instructions;

[0045] The model training module is used to perform network parameter adjustment on the motion control model based on motion instruction samples obtained by inputting the motion control model with the posture information of the robot in the training environment and the depth image captured by the image acquisition component, and motion instruction labels obtained by inputting the motion control model with the posture information of the simulated robot in the simulation environment and the attribute information of the simulation environment.

[0046] In one embodiment, the above model training module is specifically used for:

[0047] Inputting the posture information of the robot in the training environment and the depth image into the motion control model to generate motion instruction samples of the robot;

[0048] Inputting the pose information of the simulated robot in the simulated environment and the attribute information of the simulated environment into the motion control model to generate a motion instruction label of the robot;

[0049] The motion control model is network-adjusted according to the motion command sample and the motion command label.

[0050] In one of the embodiments, the motion control model is a reinforcement learning model;

[0051] The device also includes:

[0052] A parameter updating module, used for inputting the posture information of the simulated robot in the simulated environment and the attribute information of the simulated environment into the motion control model to generate motion instructions for the simulated robot;

[0053] Controlling the simulated robot to execute the motion instruction and obtaining the motion result of the simulated robot in the simulation environment;

[0054] Determining a model reward value according to the movement instruction and the movement result;

[0055] According to the model reward value, the network parameters of the motion control model are updated until a convergence condition is met.

[0056] In one embodiment, the parameter updating module is further used for:

[0057] Inputting the position and posture information of the simulated robot into a state estimation network to generate state information of the simulated robot;

[0058] Determining a network loss of the state estimation network according to the attribute information of the simulation environment and the state information of the simulation robot;

[0059] According to the network loss, the state estimation network is adjusted until a convergence condition is met.

[0060] In one embodiment, the instruction generation module is specifically used to:

[0061] Inputting the posture information of the robot into the state estimation network to generate state information of the robot;

[0062] The position information, the state information and the depth image are input into a motion control model to generate a motion instruction for the robot at the next moment.

[0063] In one embodiment, the above model training module is specifically used for:

[0064] Encoding the attribute information of the simulation environment to generate depth attribute information;

[0065] The pose information of the simulated robot in the simulated environment and the depth attribute information are input into the motion control model to generate a motion instruction label for the robot.

[0066] In one embodiment, the above model training module is specifically used for:

[0067] Determining the movement direction and terrain information of the robot according to the depth image;

[0068] The position information, the moving direction and the terrain information of the robot in the training environment are input into the motion control model to generate motion instruction samples of the robot.

[0069] In one embodiment, the above model training module is specifically used for:

[0070] Encoding the depth image to generate depth terrain information;

[0071] The posture information of the robot in the training environment and the depth terrain information are input into the motion control model to generate motion instruction samples of the robot.

[0072] In one embodiment, the attribute information of the simulated robot includes ground friction information of the simulated environment, sole contact force information of the simulated robot, and moving direction.

[0073] In one of the embodiments, during the training process of the motion control model, the simulation environment is obtained by simulating the training environment, and the training environment includes a variety of terrain environments.

[0074] According to a fourth aspect of the embodiments of the present disclosure, the present application provides a robot control device, the device comprising:

[0075] The second acquisition module is used to obtain the posture information of the robot at the current moment and the depth image collected by the image acquisition component;

[0076] A motion control module is used to input the posture information and the depth image into a motion control model to generate motion instructions for the robot and control the robot to execute the motion instructions at the next moment; wherein the motion control model is constructed based on a deep learning model, and the motion control model is used to control the robot movement so that the robot can move in a variety of terrain environments.

[0077] In a fifth aspect, the present application provides a computer program product, comprising a computer program / instructions, which implement the steps of the method described in any embodiment when the computer program / instructions are executed by a processor.

[0078] In a sixth aspect, the present application provides a robot comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in any of the above embodiments when executing the program.

[0079] In a seventh aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any of the above embodiments.

[0080] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0081] Compared with the traditional model training method, the motion control model of the robot is obtained by training based on pre-set model training data, and then the motion control model is used to control the robot movement. In the embodiment of the present disclosure, the motion instruction label is obtained by introducing the posture information of the simulated robot in the simulation environment and the attribute information of the simulation environment into the motion control model, so as to perform real-time and accurate labeling on the motion instruction samples obtained by inputting the posture information of the robot in the training environment and the depth image collected by the image acquisition component into the motion control model. The network parameters of the motion control model are adjusted accordingly, which can ensure that the amount of data for training the motion control model is sufficient, thereby improving the convergence speed of training the motion control model, and obtaining a more accurate and reasonable motion control model, which can improve the accuracy of the motion instructions of the robot generated based on the motion control model, and ensure the safety of the robot during movement.

[0082] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0084] Figure 1 is a flow chart of a robot control method shown in some exemplary embodiments.

[0085] Figure 2 is a flow chart of another robot control method shown in some exemplary embodiments.

[0086] Figure 3 is a flow chart of yet another robot control method shown in some exemplary embodiments.

[0087] Figure 4 is a flow chart of yet another robot control method shown in some exemplary embodiments.

[0088] Figure 5 is a flow chart of yet another robot control method shown in some exemplary embodiments.

[0089] Figure 6 is a block diagram of a robot control device showing some exemplary embodiments.

[0090] Figure 7 is a block diagram of another robot control device showing some exemplary embodiments.

[0091] Figure 8 is a structural diagram of a robot showing some exemplary embodiments. DETAILED DESCRIPTION

[0092] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0093] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms of "a", "said" and "the" used in this disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0094] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0095] In related technologies, the motion control model is usually trained in a training environment using a pre-set training data set based on traditional model training methods (such as model predictive control algorithms). This training method has the following problems: the training takes a long time, the accuracy of the trained model is low, and the safety of the robot cannot be guaranteed.

[0096] In order to solve the above problems, the present disclosure proposes a robot control method, which is applied in the scenario of robot motion control. Optionally, based on the motion command samples obtained by inputting the motion control model with the posture information of the robot in the training environment and the depth image collected by the image acquisition component, and the motion command labels obtained by inputting the motion control model with the posture information of the simulated robot in the simulation environment and the attribute information of the simulation environment, an accurate motion control model can be pre-trained; further, according to the posture information and depth image of the robot at the current moment, the motion command at the next moment is generated through the trained motion control model, and the robot is controlled to execute the motion command at the next moment.

[0097] Optionally, the robot control method can be executed by a motion control system that controls the movement of the robot. Exemplarily, the motion control system can be integrated locally in the robot; that is, the robot can use the motion control system to pre-train an accurate motion control model based on the motion command samples obtained by inputting the motion control model with the posture information of the robot in the training environment and the depth image collected by the image acquisition component, and the motion command labels obtained by inputting the motion control model with the posture information of the simulated robot in the simulation environment and the attribute information of the simulation environment, and then generate the motion command at the next moment through the trained motion control model according to the posture information and depth image of the robot at the current moment, and control the robot to execute the motion command at the next moment. Alternatively, the motion control system can be integrated on a server, or run on a cloud or other network server; the motion control system can be pre-trained on the cloud or other network server to obtain an accurate motion control model based on motion instruction samples obtained by inputting the motion control model with the posture information of the robot in the training environment and the depth image captured by the image acquisition component, as well as motion instruction labels obtained by inputting the motion control model with the posture information of the simulated robot in the simulation environment and the attribute information of the simulation environment; further, the robot can upload the acquired posture information and depth image of the robot at the current moment to the cloud or other network server through the network, and then generate the motion instruction for the next moment through the pre-trained motion control model, and control the robot to execute the motion instruction at the next moment.

[0098] Next, the embodiments of the present disclosure are described in detail.

[0099] The first aspect of the present disclosure provides a robot control method. Figure 1 , which comprises the following steps:

[0100] S101, obtaining the posture information of the robot and the depth image collected by the image acquisition component.

[0101] Among them, the posture information refers to the joint angles, joint angular velocities and other information of each joint of the robot at the current moment; the image acquisition component refers to the component set inside the robot for collecting images, such as a depth camera.

[0102] S102, inputting the position information and the depth image into a motion control model to generate motion instructions for the robot, and controlling the robot to execute the motion instructions.

[0103] Among them, the motion control model is trained in the following manner: at each moment, the motion control model is trained by inputting motion instruction samples obtained by inputting the motion control model based on the posture information of the robot in the training environment and the depth image collected by the image acquisition component, and the motion control model is trained by inputting the motion instruction labels obtained by inputting the motion control model based on the posture information of the simulated robot in the simulation environment and the attribute information of the simulation environment.

[0104] The motion control model refers to the model used to control the motion of the robot. The training environment refers to the actual environment for training the motion control model; the simulation environment refers to the virtual environment obtained by simulating the training environment in advance. The motion instruction refers to the specific action or movement instruction generated to control the robot at the next moment, such as the joint torque of each joint. The motion instruction sample refers to the motion instruction generated in the training environment for training the motion control model. The motion instruction label refers to the ideal motion instruction generated in the simulation environment, which is used to evaluate the motion instruction sample generated in the training environment.

[0105] Optionally, in the process of training the motion control model, the posture information of the simulated robot in the simulation environment and the attribute information of the simulation environment can be pre-input into the motion control model in the simulation environment as a teacher model to generate the motion instruction label of the robot; further, the posture information of the robot in the training environment and the depth image can be input into the motion control model in the training environment as a student model to generate the motion instruction samples of the robot; taking the motion instruction label as the standard, the motion instruction samples generated by the student model are labeled according to the motion instruction label generated by the teacher model, and then the network parameters of the motion control model are adjusted according to the difference between the motion instruction sample and the motion instruction label to obtain the final trained motion control model.

[0106] Furthermore, after obtaining the position information and depth image of the robot at the current moment, the position information and depth image can be input into the trained motion control model to obtain the motion command at the next moment, and the robot can be controlled to execute the motion command at the next moment to control the robot to move.

[0107] Compared with the traditional model training method, the motion control model of the robot is obtained by training based on pre-set model training data, and then the motion control model is used to control the robot movement. In the embodiment of the present disclosure, the motion instruction label is obtained by introducing the posture information of the simulated robot in the simulation environment and the attribute information of the simulation environment into the motion control model, so as to perform real-time and accurate labeling on the motion instruction samples obtained by inputting the posture information of the robot in the training environment and the depth image collected by the image acquisition component into the motion control model. The network parameters of the motion control model are adjusted accordingly, which can ensure that the amount of data for training the motion control model is sufficient, thereby improving the convergence speed of training the motion control model, and obtaining a more accurate and reasonable motion control model, which can improve the accuracy of the motion instructions of the robot generated based on the motion control model, and ensure the safety of the robot during movement.

[0108] Based on the above embodiments, in an exemplary embodiment, Figure 2 As shown, before using the motion control model trained in the simulation environment to train the motion control model in the training environment, the motion control model can be trained through reinforcement learning in the simulation environment. Reinforcement learning (RL) refers to a machine learning method in which an agent (i.e., the motion control model) learns to optimize the behavior strategy by interacting with the environment. For example, specific training can be performed in the following steps:

[0109] S201, inputting the posture information of the simulated robot in the simulated environment and the attribute information of the simulated environment into a motion control model to generate motion instructions for the simulated robot.

[0110] Among them, the attribute information of the simulation environment refers to the information that can be obtained in the simulation environment but cannot be obtained in the actual environment. The attribute information of the simulated robot may include the ground friction information of the simulation environment, the plantar contact force information and the moving direction of the simulated robot, etc.

[0111] Optionally, the robot can be simulated in a simulation environment to obtain a simulated robot that is the same as the robot in the training environment; the pose information of the simulated robot and the attribute information of the simulation environment are obtained, and then the acquired pose information and the attribute information of the simulation environment are input into a motion control model, and the pose information and the attribute information of the simulation environment are processed by the motion control model to generate motion instructions for the simulated robot.

[0112] S202, controlling the simulation robot to execute the motion instruction, and obtaining the motion result of the simulation robot in the simulation environment.

[0113] The motion result refers to the position, state, and other results of the simulated robot after the motion in the simulation environment.

[0114] Optionally, the simulated robot may be controlled to execute the generated motion instructions in the simulated environment, thereby obtaining the motion result of the simulated robot executing the motion instructions in the simulated environment. For example, if the motion instruction is to walk forward 0.5 meters, and there is an obstacle 0.3 meters in front of the robot in the simulated environment, the motion result obtained after the robot is controlled to execute the motion instruction may be that the robot falls or stops moving.

[0115] S203: Determine a model reward value according to the movement instruction and the movement result.

[0116] Among them, the model reward value refers to the value used to measure the effectiveness of the movement results and guide the movement control model to make adjustments.

[0117] Optionally, after obtaining the motion instructions and the motion results, the model reward value can be calculated based on the motion instructions and the motion results of the simulated robot after executing the motion instructions (such as whether the target position is reached, whether an obstacle is crossed, etc.). It should be noted that the better the motion result, the higher the model reward value; for example, if the motion result is to reach the target position, the model reward value can be determined to be a higher value, and if the motion result is not to reach the target position or the robot falls, etc., the model reward value can be determined to be a smaller value (such as a negative number).

[0118] S204: updating the network parameters of the motion control model according to the model reward value until a convergence condition is met.

[0119] The convergence condition refers to the standard that the performance or objective function of the model reaches a certain stable state during the training process. Convergence means that the training of the model has achieved the expected effect, the model strategy has become stable, and the model reward value no longer changes significantly.

[0120] Optionally, the network parameters of the motion control model can be adjusted according to the calculated model reward value. When the convergence condition is reached (for example, the model reward value is maximum), the adjustment of the parameters of the motion control model can be stopped, and the motion control model corresponding to the maximum model reward value can be determined as the trained motion control model.

[0121] In this embodiment, by setting the reinforcement learning model as the motion control model, the model reward value is determined according to the motion instructions obtained by inputting the posture information of the simulated robot and the attribute information of the simulation environment into the motion control model, and the motion results obtained by controlling the simulated robot to execute the motion instructions, and then the parameters of the motion control model are adjusted according to the model reward value until the convergence condition is reached. The motion control model can be trained more reasonably and accurately, and a more accurate motion control model can be obtained. The motion control model trained in the simulation environment is capable of serving as a teacher model to guide the motion control model serving as a student model in the training environment, thereby ensuring the accuracy of the motion control model finally trained.

[0122] Based on the above embodiments, in an exemplary embodiment, Figure 3 As shown in the figure, the process of adjusting the network parameters of the motion control model in the simulation environment also includes the following steps:

[0123] S301, inputting the posture information of the simulated robot into a state estimation network to generate state information of the simulated robot.

[0124] Among them, the state estimation network is used to estimate state information that cannot be directly collected by sensors. For example, if there is no sensor on the sole of the simulated robot's foot, the state information may include the state of the simulated robot's foot (whether it is stepping on the ground), etc.

[0125] Optionally, a state estimation network for estimating state information may be preset, and the position and posture information of the simulated robot may be input into the state estimation network to generate state information that cannot be collected by sensors.

[0126] S302: Determine the network loss of the state estimation network according to the attribute information of the simulation environment and the state information of the simulation robot.

[0127] Among them, network loss refers to the indicator used to measure the difference between the generated state information and the attribute information of the simulation environment.

[0128] Optionally, after the state information is generated, the attribute information of the simulation environment may be used as a standard to determine the network loss of the state information based on the difference between the attribute information of the simulation environment according to a preset loss function algorithm.

[0129] S303: Adjust network parameters of the state estimation network according to the network loss until a convergence condition is met.

[0130] Optionally, when training the state estimation network according to the network loss, the training will continue until the network loss is stabilized below a certain threshold, at which time the network can be considered to have converged, that is, the training process is completed, and a trained state estimation network is obtained.

[0131] In this embodiment, the network loss is determined by utilizing the state information generated by the robot's posture information and the attribute information of the simulation environment, and the state estimation network is trained based on the network loss. When the convergence conditions are met, a more reasonable and accurate state estimation network can be obtained, thereby achieving a more accurate estimation of the robot's state information.

[0132] Based on the above embodiments, in an exemplary embodiment, Figure 4 As shown, in the process of using the trained motion control model to generate the motion instruction of the robot at the next moment, the following steps may also be included:

[0133] S401, inputting the posture information of the robot into the state estimation network to generate the state information of the robot.

[0134] Optionally, the position and posture information of the robot acquired at the current moment may be input into a state estimation network, and the state information of the robot may be generated through the state estimation network.

[0135] S402, inputting the position information, the state information and the depth image into a motion control model to generate a motion instruction for the robot at a next moment.

[0136] In this embodiment, by introducing a state estimation network trained in a simulation environment and inputting the robot's posture information into the state estimation network, it is possible to accurately and efficiently generate state information that cannot be collected by sensors in the actual environment. Then, based on the posture information, state information and depth image, through the motion control model, it is possible to more accurately and efficiently generate the robot's motion instructions at the next moment, thereby achieving the effect of ensuring the safety of the robot's motion.

[0137] In an exemplary embodiment, an implementable method is provided for generating motion instruction labels for a robot, which can encode the attribute information of a simulation environment to generate depth attribute information; the posture information of the simulated robot in the simulation environment and the depth attribute information are input into the motion control model to generate the motion instruction label for the robot.

[0138] Among them, deep attribute information refers to a data representation generated after encoding the attribute information of the simulation environment, which may contain detailed features and attributes of the simulation environment. These features are converted into a format that is more suitable for model processing through certain encoding techniques (such as deep learning models or feature extraction algorithms).

[0139] Optionally, the attribute information of the simulation environment may be encoded based on a pre-set encoding technology to generate depth attribute information; further, the pose information and depth attribute information of the simulated robot may be input into a motion control model to generate a motion instruction label for the robot.

[0140] In this embodiment, the attribute information of the simulation environment is encoded to generate depth attribute information that can reflect the detailed characteristics of the simulation environment; further, the posture information of the simulated robot and these depth attribute information are input into the motion control model to generate more intelligent and accurate motion instruction labels that instruct the robot how to perform movements or task operations, thereby achieving more accurate training of the motion control model in the training environment.

[0141] In an exemplary embodiment, an implementable method is provided for generating motion command samples of a robot, wherein the moving direction and terrain information of the robot can be determined based on the depth image; the position information of the robot in the training environment, the moving direction and the terrain information are input into the motion control model to generate motion command samples of the robot.

[0142] Among them, terrain information refers to the terrain features in the training environment, such as ground undulations, slopes, obstacles, etc.; moving direction refers to the direction in which the robot moves in the environment along the optimal path, for example, walking direction, trotting direction, etc.

[0143] Optionally, the depth image can be analyzed and processed to determine the robot's moving direction and terrain information; then the robot's posture information, moving direction and terrain information in the training environment are input into the motion control model, and the motion control model can generate motion command samples based on these inputs.

[0144] In this embodiment, by extracting terrain information and movement direction from the depth image and then inputting the posture information, movement direction and terrain information into the motion control model, the robot's motion command samples can be generated more accurately. By using the motion command samples to train and optimize the robot's motion control model, the accuracy of the motion control model can be improved, enabling the robot to move effectively under different environmental conditions based on the motion control model.

[0145] In an exemplary embodiment, another feasible implementation method is provided for generating motion command samples of a robot, in which the depth image can be encoded to generate depth terrain information; the position information of the robot in the training environment and the depth terrain information are input into the motion control model to generate motion command samples of the robot.

[0146] Among them, deep terrain information is used to characterize the detailed features and properties of the training environment (such as ground friction information of the training environment), etc. These features are converted into a format that is more suitable for model processing through certain encoding techniques (such as deep learning models or feature extraction algorithms).

[0147] Optionally, the depth image of the training environment can be encoded based on a pre-set encoding technology to generate depth terrain information; further, the robot's posture information and depth terrain information can be input into a motion control model to generate motion command samples for the robot.

[0148] In this embodiment, by encoding the depth image of the training environment, depth terrain information that can reflect the detailed features of the training environment is generated; further, the robot's posture information and these depth terrain information are input into the motion control model to generate more accurate motion command samples of the robot, thereby achieving the effect of more efficient and accurate training of the motion control model using motion command samples in the training environment.

[0149] A second aspect of the present disclosure provides a robot control method. Figure 5 , which comprises the following steps:

[0150] S501, obtaining the posture information of the robot at the current moment and the depth image collected by the image acquisition component.

[0151] S502, inputting the position information and the depth image into a motion control model to generate a motion instruction for the robot, and controlling the robot to execute the motion instruction at the next moment.

[0152] Among them, the motion control model is built based on a deep learning model, and the motion control model is used to control the movement of the robot so that the robot can move in a variety of terrain environments.

[0153] Optionally, motion command samples obtained by inputting the posture information of the robot in the training environment and the depth images captured by the image acquisition component into the deep learning model can be obtained in advance, as well as motion command labels obtained by inputting the posture information of the simulated robot in the simulation environment and the attribute information of the simulation environment into the deep learning model. Then, based on the motion command samples and motion command labels, the network parameters of the deep learning model are adjusted to construct a motion control model of the robot.

[0154] Furthermore, after the motion control model is constructed, the position information of the robot at the current moment and the depth image collected by the graphics acquisition component can be input into the motion control model to generate the motion command of the robot at the next moment; then the robot is controlled to execute the motion command at the next moment, that is, the robot is controlled to move, so that the robot can move effectively and safely in the face of various terrain environments (such as flat ground, ramps, obstacles, etc.).

[0155] In this embodiment, by constructing a motion control model of the robot based on a deep learning model and using the motion control model to control the robot to move, the robot's ability to adapt to complex environments can be improved, enabling it to autonomously navigate and move under various terrain conditions, thereby ensuring the accuracy and safety of the robot during movement.

[0156] Corresponding to the embodiments of the aforementioned method, the present disclosure also provides embodiments of a device and a terminal to which the device is applied.

[0157] A third aspect of the present disclosure provides a robot control device. Figure 6 , the above-mentioned robot control device comprises:

[0158] The first acquisition module 601 is used to acquire the posture information of the robot at the current moment and the depth image acquired by the image acquisition component;

[0159] An instruction generation module 602, used to input the posture information and the depth image into a motion control model to generate a motion instruction for the robot, and control the robot to execute the motion instruction at the next moment;

[0160] The model training module 603 is used to adjust the network parameters of the motion control model at each moment based on the motion instruction samples obtained by inputting the motion control model with the posture information of the robot in the training environment and the depth image captured by the image acquisition component, and the motion instruction labels obtained by inputting the motion control model with the posture information of the simulated robot in the simulation environment and the attribute information of the simulation environment.

[0161] In one embodiment, the model training module 603 is specifically used for:

[0162] Inputting the posture information of the robot in the training environment and the depth image into the motion control model to generate motion instruction samples of the robot;

[0163] Inputting the pose information of the simulated robot in the simulated environment and the attribute information of the simulated environment into the motion control model to generate a motion instruction label of the robot;

[0164] The motion control model is network-adjusted according to the motion command sample and the motion command label.

[0165] In one of the embodiments, the motion control model is a reinforcement learning model;

[0166] Before the model training module, the device also includes:

[0167] A parameter updating module, used for inputting the posture information of the simulated robot in the simulated environment and the attribute information of the simulated environment into the motion control model to generate motion instructions for the simulated robot;

[0168] Controlling the simulated robot to execute the motion instruction and obtaining the motion result of the simulated robot in the simulation environment;

[0169] Determining a model reward value according to the movement instruction and the movement result;

[0170] According to the model reward value, the network parameters of the motion control model are updated until a convergence condition is met.

[0171] In one embodiment, the parameter updating module is further used for:

[0172] Inputting the position and posture information of the simulated robot into a state estimation network to generate state information of the simulated robot;

[0173] Determining a network loss of the state estimation network according to the attribute information of the simulation environment and the state information of the simulation robot;

[0174] According to the network loss, the state estimation network is adjusted until a convergence condition is met.

[0175] In one embodiment, the instruction generation module 602 is specifically used to:

[0176] Inputting the posture information of the robot into the state estimation network to generate state information of the robot;

[0177] The position information, the state information and the depth image are input into a motion control model to generate a motion instruction for the robot at the next moment.

[0178] In one embodiment, the model training module 603 is specifically used for:

[0179] Encoding the attribute information of the simulation environment to generate depth attribute information;

[0180] The pose information of the simulated robot in the simulated environment and the depth attribute information are input into the motion control model to generate a motion instruction label for the robot.

[0181] In one embodiment, the model training module 603 is specifically used for:

[0182] Determining the movement direction and terrain information of the robot according to the depth image;

[0183] The position information, the moving direction and the terrain information of the robot in the training environment are input into the motion control model to generate motion instruction samples of the robot.

[0184] In one embodiment, the model training module 603 is specifically used for:

[0185] Encoding the depth image to generate depth terrain information;

[0186] The posture information of the robot in the training environment and the depth terrain information are input into the motion control model to generate motion instruction samples of the robot.

[0187] In one embodiment, the attribute information of the simulated robot includes ground friction information of the simulated environment, sole contact force information of the simulated robot, and moving direction.

[0188] In one of the embodiments, during the training process of the motion control model, the simulation environment is obtained by simulating the training environment, and the training environment includes a variety of terrain environments.

[0189] A fourth aspect of the present disclosure provides a robot control device. Figure 7 , the above-mentioned robot control device comprises:

[0190] The second acquisition module 701 is used to acquire the posture information of the robot at the current moment and the depth image acquired by the image acquisition component;

[0191] The motion control module 702 is used to input the posture information and the depth image into the motion control model to generate the motion command of the robot and control the robot to execute the motion command at the next moment; wherein, the motion control model is constructed based on the deep learning model, and the motion control model is used to control the robot movement so that the robot can move in a variety of terrain environments.

[0192] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, which will not be repeated here.

[0193] A fifth aspect of the present disclosure provides a computer program product, including a computer program / instruction, which implements the method described in any of the above aspects when executed by a processor.

[0194] For the device embodiments and computer program product embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial description of the method embodiments. In addition, the device embodiments described above are merely schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the disclosed solution. Those of ordinary skill in the art can understand and implement it without paying creative labor.

[0195] In a sixth aspect, some embodiments of the present disclosure provide a robot, see Figure 8 , which shows the structure of the robot, the robot includes a memory and a processor, the memory is used to store computer instructions that can be executed on the processor, and the processor is used to control the robot based on any of the methods described in any of the above aspects when executing the computer instructions.

[0196] In a seventh aspect, the present disclosure, in an exemplary embodiment, further provides a non-transitory computer-readable storage medium including instructions, such as a memory including instructions, and the instructions can be executed by a processor of a robot or a computer device to complete the robot control method of the robot. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0197] The above describes specific embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0198] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the inventions claimed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not claimed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0199] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

[0200] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the scope of protection of the present disclosure.

Claims

1. A robot control method, characterized in that: The method comprises: Obtaining the robot's posture information and the depth image collected by the image acquisition component; Inputting the position information and the depth image into a motion control model to generate motion instructions for the robot, and controlling the robot to execute the motion instructions; The motion control model is trained in the following way: Based on the motion command samples obtained by inputting the motion control model with the posture information of the robot in the training environment and the depth image collected by the image acquisition component, and the motion command labels obtained by inputting the motion control model with the posture information of the simulated robot in the simulation environment and the attribute information of the simulation environment, the network parameters of the motion control model are adjusted; Wherein, the motion control model is a reinforcement learning model; The method further comprises: Inputting the pose information of the simulated robot in the simulated environment and the attribute information of the simulated environment into a motion control model to generate motion instructions for the simulated robot; Controlling the simulated robot to execute the motion instruction and obtaining the motion result of the simulated robot in the simulation environment; Determining a model reward value according to the movement instruction and the movement result; According to the model reward value, the network parameters of the motion control model are updated until a convergence condition is met; The method further comprises: Inputting the pose information of the simulated robot into a state estimation network to generate state information of the simulated robot, wherein the state information is used to characterize attribute information of the simulated environment; Determining a network loss of the state estimation network according to the attribute information of the simulation environment and the state information of the simulation robot; According to the network loss, adjusting the network parameters of the state estimation network until a convergence condition is met; The step of inputting the position information and the depth image into a motion control model to generate motion instructions for the robot includes: Inputting the posture information of the robot into the state estimation network to generate state information of the robot; The position information, the state information and the depth image are input into a motion control model to generate a motion instruction for the robot at the next moment.

2. The method according to claim 1, characterized in that The motion control model is adjusted network parameters based on the motion command samples obtained by inputting the motion control model based on the posture information of the robot in the training environment and the depth image collected by the image acquisition component, and the motion command labels obtained by inputting the posture information of the simulated robot in the simulation environment and the attribute information of the simulation environment into the motion control model, including: Inputting the posture information of the robot in the training environment and the depth image into the motion control model to generate motion instruction samples of the robot; Inputting the pose information of the simulated robot in the simulated environment and the attribute information of the simulated environment into the motion control model to generate a motion instruction label of the robot; The motion control model is network-adjusted according to the motion command sample and the motion command label.

3. The method according to claim 2, characterized in that The step of inputting the pose information of the simulated robot in the simulated environment and the attribute information of the simulated environment into the motion control model to generate the motion instruction label of the robot comprises: Encoding the attribute information of the simulation environment to generate depth attribute information; The pose information of the simulated robot in the simulated environment and the depth attribute information are input into the motion control model to generate a motion instruction label for the robot.

4. The method according to claim 2, characterized in that: The step of inputting the posture information of the robot in the training environment and the depth image into the motion control model to generate motion instruction samples of the robot includes: Determining the movement direction and terrain information of the robot according to the depth image; The position information, the moving direction and the terrain information of the robot in the training environment are input into the motion control model to generate motion instruction samples of the robot.

5. The method according to claim 2, characterized in that: The step of inputting the posture information of the robot in the training environment and the depth image into the motion control model to generate motion instruction samples of the robot includes: Encoding the depth image to generate depth terrain information; The posture information of the robot in the training environment and the depth terrain information are input into the motion control model to generate motion instruction samples of the robot.

6. The method according to claim 1, characterized in that The attribute information of the simulation robot includes ground friction information of the simulation environment, sole contact force information of the simulation robot, and moving direction.

7. The method according to claim 1, characterized in that The simulation environment is obtained by simulating the training environment, and the training environment includes a variety of terrain environments.

8. A robot control method, characterized in that: The method comprises: Obtain the robot's current position information and the depth image collected by the image acquisition component; Inputting the position information and the depth image into a motion control model to generate a motion instruction for the robot, and controlling the robot to execute the motion instruction at the next moment; Wherein, the motion control model is constructed based on a deep learning model, and the motion control model is used to control the motion of the robot so that the robot can move in a variety of terrain environments; The motion control model is trained in the following way: Based on the motion command samples obtained by inputting the motion control model with the posture information of the robot in the training environment and the depth image collected by the image acquisition component, and the motion command labels obtained by inputting the motion control model with the posture information of the simulated robot in the simulation environment and the attribute information of the simulation environment, the network parameters of the motion control model are adjusted; Wherein, the motion control model is a reinforcement learning model; The method further comprises: Inputting the pose information of the simulated robot in the simulated environment and the attribute information of the simulated environment into a motion control model to generate motion instructions for the simulated robot; Controlling the simulated robot to execute the motion instruction and obtaining the motion result of the simulated robot in the simulation environment; Determining a model reward value according to the movement instruction and the movement result; According to the model reward value, the network parameters of the motion control model are updated until a convergence condition is met; The method further comprises: Inputting the pose information of the simulated robot into a state estimation network to generate state information of the simulated robot, wherein the state information is used to characterize attribute information of the simulated environment; Determining a network loss of the state estimation network according to the attribute information of the simulation environment and the state information of the simulation robot; According to the network loss, adjusting the network parameters of the state estimation network until a convergence condition is met; The step of inputting the position information and the depth image into a motion control model to generate motion instructions for the robot includes: Inputting the posture information of the robot into the state estimation network to generate state information of the robot; The position information, the state information and the depth image are input into a motion control model to generate a motion instruction for the robot at the next moment.

9. A robot control device, characterized in that: The device comprises: A first acquisition module is used to acquire the posture information of the robot and the depth image acquired by the image acquisition component; An instruction generation module, used for inputting the posture information and the depth image into a motion control model to generate motion instructions for the robot, and controlling the robot to execute the motion instructions; A model training module, for performing network parameter adjustment on the motion control model based on motion instruction samples obtained by inputting the motion control model with the posture information of the robot in the training environment and the depth image collected by the image acquisition component, and motion instruction labels obtained by inputting the motion control model with the posture information of the simulated robot in the simulation environment and the attribute information of the simulation environment; Wherein, the motion control model is a reinforcement learning model; The device also includes: A parameter updating module, used for inputting the posture information of the simulated robot in the simulated environment and the attribute information of the simulated environment into the motion control model to generate motion instructions for the simulated robot; controlling the simulated robot to execute the motion instructions and obtaining the motion results of the simulated robot in the simulated environment; determining a model reward value according to the motion instructions and the motion results; and updating the network parameters of the motion control model according to the model reward value until a convergence condition is met; The device also includes: A network parameter adjustment module, used for inputting the posture information of the simulated robot into a state estimation network to generate state information of the simulated robot, wherein the state information is used to characterize the attribute information of the simulation environment; determining the network loss of the state estimation network according to the attribute information of the simulation environment and the state information of the simulated robot; and adjusting the network parameters of the state estimation network according to the network loss until a convergence condition is met; The motion control model is used to: Inputting the posture information of the robot into the state estimation network to generate state information of the robot; The position information, the state information and the depth image are input into a motion control model to generate a motion instruction for the robot at the next moment.

10. A robot control device, characterized in that: The device comprises: The second acquisition module is used to obtain the posture information of the robot at the current moment and the depth image collected by the image acquisition component; A motion control module, used for inputting the position information and the depth image into a motion control model to generate a motion instruction for the robot, and controlling the robot to execute the motion instruction at the next moment; Wherein, the motion control model is constructed based on a deep learning model, and the motion control model is used to control the motion of the robot so that the robot can move in a variety of terrain environments; A model training module, for performing network parameter adjustment on the motion control model based on motion instruction samples obtained by inputting the motion control model with the posture information of the robot in the training environment and the depth image collected by the image acquisition component, and motion instruction labels obtained by inputting the motion control model with the posture information of the simulated robot in the simulation environment and the attribute information of the simulation environment; Wherein, the motion control model is a reinforcement learning model; The device also includes: A parameter updating module, used for inputting the posture information of the simulated robot in the simulated environment and the attribute information of the simulated environment into the motion control model to generate motion instructions for the simulated robot; controlling the simulated robot to execute the motion instructions and obtaining the motion results of the simulated robot in the simulated environment; determining a model reward value according to the motion instructions and the motion results; and updating the network parameters of the motion control model according to the model reward value until a convergence condition is met; The device also includes: A network parameter adjustment module, used for inputting the posture information of the simulated robot into a state estimation network to generate state information of the simulated robot, wherein the state information is used to characterize the attribute information of the simulation environment; determining the network loss of the state estimation network according to the attribute information of the simulation environment and the state information of the simulated robot; and adjusting the network parameters of the state estimation network according to the network loss until a convergence condition is met; The motion control module is used for: Inputting the posture information of the robot into the state estimation network to generate state information of the robot; The position information, the state information and the depth image are input into a motion control model to generate a motion instruction for the robot at the next moment.

11. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

12. A robot, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images