Robot control method and apparatus, and electronic device, computer-readable storage medium and computer program product

Abstract

Provided in the present application are a robot control method and apparatus, and an electronic device, a storage medium and a program product. The method comprises: acquiring state data for indicating the current motion state of a robot, and acquiring environment data for indicating the environment where the robot is currently located; on the basis of the state data, predicting an initial action parameter for controlling the robot to imitate an object action of a target object; on the basis of the environment data, predicting an environment influence parameter for representing the influence of the environment on the robot imitating the object action; fusing the initial action parameter with the environment influence parameter to obtain a fused action parameter of the robot; and generating a control instruction on the basis of the fused action parameter, wherein the control instruction is used for controlling the robot to execute a target action indicated by the fused action parameter.

Description

Robot control method, device, electronic device, computer-readable storage medium, and computer program product

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based on the Chinese patent application with application number 202310960451.9 and application date July 31, 2023, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is hereby introduced into this application as a reference. Technical Field

[0003] The present application relates to the field of computer technology, and in particular to a robot control method, device, electronic device, computer-readable storage medium, and computer program product. Background Art

[0004] Artificial Intelligence (AI) is the theory, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. AI technology is an interdisciplinary discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics.

[0005] In related technologies, robots are usually controlled by writing fixed program instructions. This results in the controlled robot's movements being unable to accurately adapt to the robot's environment, resulting in poor robot motion performance.

[0006] Summary of the Invention

[0007] The embodiments of the present application provide a robot control method, device, electronic device, computer-readable storage medium, and computer program product, which can effectively improve the robot's motion performance.

[0008] The technical solution of the embodiment of the present application is implemented as follows:

[0009] The present invention provides a robot control method, including:

[0010] Acquiring state data indicating a current motion state of the robot and acquiring environmental data indicating a current environment of the robot;

[0011] predicting, based on the state data, initial motion parameters for controlling the robot to imitate an object motion of a target object;

[0012] Based on the environmental data, predicting an environmental impact parameter for characterizing the impact of the environment on the robot's imitation of the object's action;

[0013] Fusing the initial motion parameters and the environmental impact parameters to obtain fused motion parameters of the robot;

[0014] Based on the fused action parameters, a control instruction is generated, where the control instruction is used to control the robot to execute a target action indicated by the fused action parameters.

[0015] An embodiment of the present application provides a robot control device, comprising:

[0016] a feature acquisition module configured to acquire state data indicating a current motion state of the robot and to acquire environmental data indicating a current environment in which the robot is located;

[0017] an initial motion prediction module configured to predict initial motion parameters for controlling the robot to imitate an object motion of a target object based on the state data;

[0018] an offset action prediction module configured to predict, based on the environmental data, an environmental impact parameter used to characterize the impact of the environment on the robot's imitation of the object's action;

[0019] a fusion module configured to fuse the initial motion parameters and the environmental impact parameters to obtain fused motion parameters of the robot;

[0020] A generation module is configured to generate a control instruction based on the fusion action parameter, where the control instruction is used to control the robot to perform a target action indicated by the fusion action parameter.

[0021] An embodiment of the present application provides an electronic device, including:

[0022] a memory configured to store computer-executable instructions or computer programs;

[0023] The processor is configured to implement the robot control method provided in the embodiment of the present application when executing the computer executable instructions or computer program stored in the memory.

[0024] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions configured to cause a processor to execute the instructions to implement the robot control method provided in the embodiment of the present application.

[0025] An embodiment of the present application provides a computer program product, which includes a computer program or computer-executable instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the robot control method described above in the embodiment of the present application.

[0026] The embodiments of the present application have the following beneficial effects:

[0027] By predicting initial motion parameters for controlling the robot to imitate the target object's motion, and predicting environmental influence parameters for characterizing the impact of the environment on the robot's imitation of the target object's motion, the initial motion parameters and the environmental influence parameters are fused to obtain the robot's fused motion parameters. Since the obtained environmental influence parameters are highly adaptable to the environment, the robot can highly adapt to the environment and perform motions. The obtained initial motion parameters are highly able to imitate the target object's motion, allowing the robot to highly simulate the target object. Consequently, the obtained fused motion parameters are both highly adaptable to the environment and highly simulate the target object. By generating control instructions based on the fused motion parameters, the robot is controlled to move according to the target motion indicated by the fused motion parameters, so that the robot's motion is both highly adaptable to the environment and highly simulates the target object, effectively improving the robot's motion performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] FIG1 is a schematic diagram of the architecture of a robot control system provided in an embodiment of the present application;

[0029] FIG2 is a schematic diagram of the structure of an electronic device for robot control provided in an embodiment of the present application;

[0030] FIG3 is a flowchart of a robot control method according to an embodiment of the present application;

[0031] FIG4 is a second flow chart of the robot control method provided in an embodiment of the present application;

[0032] FIG5 is a schematic diagram of the network structure of the initial feature prediction network provided by an embodiment of the present application;

[0033] FIG6 is a third flow chart of the robot control method provided in an embodiment of the present application;

[0034] FIG7 is a fourth flow chart of the robot control method provided in an embodiment of the present application;

[0035] FIG8 is a fifth flow chart of the robot control method provided in an embodiment of the present application;

[0036] FIG9 is a schematic diagram of the network structure of the environmental impact parameter prediction network provided in an embodiment of the present application;

[0037] FIG10 is a sixth flow chart of the robot control method provided in an embodiment of the present application;

[0038] FIG11 is a seventh flow chart of a robot control method according to an embodiment of the present application;

[0039] FIG12 is a flowchart of a robot control method according to an embodiment of the present application;

[0040] FIG13 is a ninth flowchart of a robot control method according to an embodiment of the present application;

[0041] FIG14 is a flowchart diagram 10 of a robot control method according to an embodiment of the present application;

[0042] FIG15 is a flowchart diagram 11 of a robot control method according to an embodiment of the present application;

[0043] FIG16 is a flowchart diagram 12 of the robot control method provided in an embodiment of the present application;

[0044] FIG17 is a first schematic diagram of the principle of the robot control method provided in an embodiment of the present application;

[0045] FIG18 is a second schematic diagram of the principle of the robot control method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limiting this application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0047] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0048] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0050] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0051] 1) In response to: used to indicate the conditions or states on which the executed operations depend. When the dependent conditions or states are met, one or more operations executed can be in real time or with a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations executed are executed.

[0052] 2) Artificial Intelligence (AI): This refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. AI technology is an interdisciplinary discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics.

[0053] 3) Convolutional Neural Networks (CNN): These are a type of feed-forward neural network (FNN) with a deep structure that incorporates convolutional computations. They are a representative algorithm for deep learning. CNNs possess representation learning capabilities and can perform shift-invariant classification of input images based on their hierarchical structure.

[0054] 4) Machine Learning (ML): This is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is at the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications span all areas of AI. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0055] 5) Convolutional Layer: Each convolutional layer in a convolutional neural network consists of several convolutional units, whose parameters are optimized through a backpropagation algorithm. The purpose of the convolution operation is to extract different features from the input. The first convolutional layer may only extract low-level features such as edges, lines, and corners. Networks with more layers can iteratively extract more complex features from these low-level features.

[0056] 6) Reinforcement Learning (RL): Also known as reinforcement learning, evaluation learning, or enhanced learning, it is a paradigm and methodology in machine learning that describes and solves the problem of an intelligent agent learning strategies to maximize rewards or achieve specific goals during its interaction with the environment. A common model for RL is the standard Markov decision process (MDP). Depending on the given conditions, RL can be divided into model-based RL and model-free RL, as well as active RL and passive RL. Variants of RL include inverse reinforcement learning, hierarchical RL, and RL for partially observable systems. Algorithms used to solve RL problems can be categorized into policy search algorithms and value function algorithms. Deep learning models can be applied to RL, forming deep RL. RL theory, inspired by behaviorist psychology, focuses on online learning and attempts to maintain a balance between exploration and exploitation. Unlike supervised and unsupervised learning, reinforcement learning does not require any pre-defined data. Instead, it acquires learning information and updates model parameters by receiving rewards (feedback) from the environment in response to actions. Reinforcement learning has been discussed in fields such as information theory, game theory, and automatic control. It has been used to explain equilibrium states under bounded rationality, design recommendation systems, and robotic interaction systems. Some complex reinforcement learning algorithms possess, to a certain extent, general intelligence for solving complex problems.

[0057] 7) Robot: A machine that can be programmed and automatically controlled to perform tasks such as movement or movement. Possessing fundamental capabilities such as perception, decision-making, and execution, robots can assist or even replace humans in completing dangerous, arduous, and complex tasks, improving work efficiency and quality, contributing to human well-being and expanding the scope of human activities and capabilities.

[0058] 8) Control Theory: Control theory is the discipline that studies the theories, methods, techniques, and engineering applications of control. Based on cybernetics, systems theory, and information theory, control science examines common problems across various application fields, specifically how to model a system, analyze its internal and environmental information, and adopt appropriate control and decision-making behaviors to achieve control objectives. This close integration with various application fields has led to the rich and diverse content of control engineering.

[0059] 9) Legged Robot: A legged robot is a robot that possesses both wheeled and legged locomotion. Wheeled locomotion refers to the robot's movement driven by wheels that are in contact with the ground. Legged locomotion refers to the robot having a leg structure, such as two, four, or any other number of legs, that achieves walking motion by alternating leg lifts.

[0060] During the implementation of the embodiments of this application, the applicant discovered that the related technology has the following problems:

[0061] In related technologies, robots are usually controlled by writing fixed program instructions. This results in the controlled robot's movements being unable to accurately adapt to the robot's environment, resulting in poor robot motion performance.

[0062] Embodiments of the present application provide a robot control method, device, electronic device, computer-readable storage medium, and computer program product, which can effectively improve the motion performance of the robot. The following describes an exemplary application of the robot control system provided by the embodiments of the present application.

[0063] Refer to Figure 1, which is a schematic diagram of the architecture of the robot control system 100 provided in an embodiment of the present application. The terminal (terminal 400 is shown as an example) is connected to the server 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0064] The terminal 400 is configured to allow a user to use a client 410, and to display a control interface on a graphical interface 410-1 (graphic interface 410-1 is shown as an example). The terminal 400 and the server 200 are connected to each other via a wired or wireless network.

[0065] In some embodiments, the server 200 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and basic cloud computing services such as big data and artificial intelligence platforms. The terminal 400 can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart TV, a smart watch, a car terminal, etc., but is not limited to this. The electronic device provided in the embodiment of the present application can be implemented as a terminal or as a server. The terminal and the server can be directly or indirectly connected by wired or wireless communication, which is not limited in the embodiment of the present application.

[0066] In some embodiments, the server 200 obtains status data and environmental data, determines initial action parameters and environmental impact parameters based on the environmental data and status data, fuses the initial action parameters and environmental impact parameters to obtain fused action parameters, and sends the fused action parameters to the terminal 400. The terminal 400 generates control instructions based on the fused action parameters.

[0067] In other embodiments, the terminal 400 obtains status data and environmental data, determines initial action parameters and environmental impact parameters based on the environmental data and status data, fuses the initial action parameters and environmental impact parameters to obtain fused action parameters, and generates control instructions based on the fused action parameters.

[0068] In other embodiments, the server 200 obtains status data and environmental data, determines initial action parameters and environmental impact parameters based on the environmental data and status data, fuses the initial action parameters and environmental impact parameters to obtain fused action parameters, generates control instructions based on the fused action parameters, and sends the control instructions to the terminal 400.

[0069] In other embodiments, the embodiments of the present application can be implemented with the help of cloud technology. Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and network within a wide area network or local area network to realize data calculation, storage, processing, and sharing.

[0070] Cloud technology is a general term for network, information, integration, management platform, and application technologies used in the cloud computing business model. It can form a resource pool that can be used flexibly and conveniently on demand. Cloud computing technology will become a key support. The backend services of technical network systems require a large amount of computing and storage resources.

[0071] Referring to FIG. 2 , FIG. 2 is a schematic diagram of the structure of an electronic device 500 for robot control provided in an embodiment of the present application, wherein the electronic device 500 shown in FIG. 2 may be the server 200 or the terminal 400 in FIG. 1 , and the electronic device 500 shown in FIG. 2 includes: at least one processor 430, a memory 450, and at least one network interface 420. The various components in the electronic device 500 are coupled together via a bus system 440. It is understood that the bus system 440 is configured to enable connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, all the various buses are labeled as the bus system 440 in FIG.

[0072] The processor 430 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0073] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 430.

[0074] The memory 450 includes volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0075] In some embodiments, the memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplified below.

[0076] Operating system 451, including system programs configured to handle various basic system services and perform hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., configured to implement various basic services and handle hardware-based tasks;

[0077] The network communication module 452 is configured to reach other electronic devices via one or more (wired or wireless) network interfaces 420. Exemplary network interfaces 420 include: Bluetooth, Wireless Fidelity (WiFi), and Universal Serial Bus (USB).

[0078] In some embodiments, the robot control device provided in the embodiments of the present application can be implemented using software. FIG2 shows a robot control device 455 stored in memory 450. The robot control device 455 can be software in the form of a program or plug-in, and includes the following software modules: a feature acquisition module 4551, an initial motion prediction module 4552, an initial motion prediction module 4553, a fusion module 4554, and a generation module 4555. These modules are logical and can be arbitrarily combined or further separated according to the functions implemented. The functions of each module will be described below.

[0079] In other embodiments, the robot control device provided in the embodiments of the present application can be implemented in hardware. As an example, the robot control device provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the robot control method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0080] In some embodiments, the terminal or server can implement the robot control method provided in the embodiments of the present application by running a computer program or computer executable instructions. For example, the computer program can be a native program in the operating system (for example, a dedicated robot control program) or a software module, for example, a robot control module that can be embedded in any program (such as an instant messaging client, a photo album program, an electronic map client, a navigation client); for example, it can be a native application (APP, Application), that is, a program that needs to be installed in the operating system to run. In short, the above-mentioned computer program can be any form of application, module or plug-in.

[0081] The robot control method provided in the embodiment of the present application will be explained in combination with the exemplary application and implementation of the server or terminal provided in the embodiment of the present application.

[0082] Refer to Figure 3, which is a flow chart of the robot control method provided in an embodiment of the present application. It will be explained in conjunction with steps 101 to 105 shown in Figure 3. The robot control method provided in an embodiment of the present application can be implemented by a server or a terminal alone, or by a server and a terminal in collaboration. The following will be explained using the server alone as an example.

[0083] In step 101, state data indicating the current motion state of the robot is obtained, and environment data indicating the current environment of the robot is obtained.

[0084] In some embodiments, the above-mentioned acquisition of status data for indicating the current motion state of the robot can be achieved in the following manner: acquiring status information collected by a status sensor provided on the robot, the status information being used to indicate the current motion state of the robot; performing feature extraction on the status information to obtain status data.

[0085] In some embodiments, the aforementioned state information refers to data or information regarding the robot's current motion or operational state. This state information may include the robot's position, speed, direction, posture, and any other relevant parameters or metrics. State information is provided by state sensors and indicates the robot's current task, action, or behavior. Analyzing and understanding this state information can help the robot system adjust and control its motion in real time to adapt to different environments, tasks, or requirements.

[0086] In some embodiments, the above-mentioned state information may be the current motion parameters of the robot, which include the position of the robot's end effector, the position and rotation angle of the heel joint, the linear velocity and angular velocity of the root joint, etc. The above-mentioned state data is in the vector form corresponding to the state information.

[0087] In some embodiments, the feature extraction refers to a process of converting state information in parameter form into state data in vector form.

[0088] In some embodiments, the state sensor refers to a sensor used to collect state information of the robot, such as an angular velocity sensor, a linear velocity sensor, and an angle sensor.

[0089] In some embodiments, the above-mentioned acquisition of environmental data used to indicate the current environment of the robot can be achieved in the following way: obtaining environmental information collected by environmental sensors provided on the robot, the environmental information is used to indicate the current environment of the robot; performing feature extraction on the environmental information to obtain environmental data.

[0090] In some embodiments, the above-mentioned environmental information may be information about the robot's current environment, for example, whether there are obstacles ahead of the robot, the robot's altitude, the road conditions ahead of the robot, etc.

[0091] In some embodiments, the feature extraction refers to a process of converting state information in parameter form into environmental data in vector form.

[0092] In some embodiments, the above-mentioned environmental sensor refers to a sensor used to collect environmental information of the robot, such as a distance sensor, an obstacle detection sensor, etc.

[0093] In step 102 , initial motion parameters for controlling the robot to imitate an object motion of a target object are predicted based on the state data.

[0094] In some embodiments, the initial motion parameters are used for the robot to imitate the motion of the target object.

[0095] In some embodiments, the initial action parameters are predicted by an initial feature prediction network, which includes an initial data adjustment layer and an initial parameter prediction layer.

[0096] In some embodiments, the above-mentioned initial feature prediction network has the ability to imitate the action of the target object, and the initial feature prediction network can be trained using the action data of the target object, so that the trained initial feature prediction network has the ability to imitate the action of the target object.

[0097] As an example, when the above-mentioned robot is a quadruped robot, the target object imitated by the quadruped robot can be quadrupeds such as "dog", "monkey", and "rabbit". The initial motion parameters of the quadruped robot are predicted through the environmental data and state data of the quadruped robot. The obtained initial motion parameters of the robot are highly similar to the movement posture of quadrupeds such as "dog", "monkey", and "rabbit", so that the quadruped robot can imitate the movements of quadrupeds such as "dog", "monkey", and "rabbit".

[0098] As an example, when the above-mentioned robot is an eight-legged robot, the target object imitated by the eight-legged robot can be an eight-legged animal such as a "crab" and a "spider". The initial motion parameters of the eight-legged robot are predicted through the environmental data and state data of the eight-legged robot. The obtained initial motion parameters of the robot are highly similar to the movement posture of eight-legged animals such as "crabs" and "spiders", so that the eight-legged robot can imitate the movements of eight-legged animals such as "crabs" and "spiders".

[0099] In some embodiments, referring to FIG. 4 , FIG. 4 is a second flow chart of the robot control method provided in an embodiment of the present application. Step 102 shown in FIG. 3 can be implemented through steps 1021 to 1022 shown in FIG. 4 .

[0100] In step 1021, the initial data adjustment layer is called to perform data adjustment on the state data to obtain initial reference data.

[0101] In some embodiments, the above-mentioned data adjustment refers to the process of performing feature transformation on the state data to obtain initial reference data.

[0102] As an example, referring to Figure 5, which is a schematic diagram of the network structure of the initial feature prediction network provided by an embodiment of the present application, the initial data adjustment layer 51 is called to adjust the state data a1 to obtain the initial reference data a2.

[0103] In step 1022, the initial parameter prediction layer is called to predict the robot's motion parameters based on the initial reference data and environmental data to obtain the robot's initial motion parameters.

[0104] In some embodiments, the above-mentioned motion parameter prediction refers to a process for predicting the initial motion parameters of the robot.

[0105] As an example, referring to FIG5 , the initial parameter prediction layer 52 is called to predict the motion parameters of the robot based on the initial reference data a2 and the environmental data a3 to obtain the initial motion parameters at of the robot.

[0106] In this way, by predicting the initial motion parameters of the robot based on state data and environmental data, the initial motion parameters of the robot are obtained, so that the obtained initial motion parameters can highly imitate the motion of the target object, thereby enabling the robot to highly simulate the target object.

[0107] In some embodiments, referring to FIG6 , FIG6 is a flow chart of the robot control method provided in an embodiment of the present application. Before step 102 shown in FIG3 , the initial feature prediction network can also be trained by executing steps 106 to 108 as follows.

[0108] In step 106 , object state data indicating the motion state of the target object and tag motion parameters of the target object are acquired.

[0109] In some embodiments, the above-mentioned acquisition of object state data for indicating the motion state of the target object can be achieved by: obtaining object state information for indicating the motion state of the target object collected by a state sensor set on the target object, performing feature extraction on the above-mentioned object state information, and obtaining object state data.

[0110] As an example, when the target object is a "dog", the object state information indicating the movement state of the "dog" collected by the state sensor set on the "dog" is obtained, and features are extracted from the above object state information to obtain the object state data of the "dog".

[0111] In step 107, the initial feature prediction network is called to predict the action parameters of the target object based on the object state data to obtain the predicted action parameters of the target object.

[0112] In some embodiments, referring to FIG. 7 , FIG. 7 is a fourth flow chart of the robot control method provided in an embodiment of the present application. Step 107 shown in FIG. 6 can be implemented through steps 1071 to 1073 shown in FIG. 7 .

[0113] In step 1071, the object state data and the state data are fused to obtain state fusion data.

[0114] As an example, referring to FIG5 , the object state data a4 and the state data a5 are fused to obtain state fusion data a6 .

[0115] In step 1072, the initial data adjustment layer is called to perform data adjustment on the state fusion data to obtain initial adjusted state data.

[0116] As an example, referring to FIG5 , the initial data adjustment layer 51 is called to perform data adjustment on the state fusion data a6 to obtain initial adjusted state data a7.

[0117] In step 1073, the initial parameter prediction layer is called to predict the action parameters of the target object based on the initial adjustment state data and the state data to obtain the predicted action parameters of the target object.

[0118] As an example, referring to FIG5 , the initial parameter prediction layer 52 is called to predict the action parameters of the target object based on the initial adjustment state data a7 and the state data a1 to obtain the predicted action parameters a8 of the target object.

[0119] In this way, fusing object state data with status data integrates different types of information, enhancing the richness and expressiveness of features. This helps improve the performance and accuracy of subsequent steps. By adjusting the initial features to make them more suitable for the target task or scenario, this helps resolve mismatches or deviations between features, thereby improving the accuracy of subsequent motion parameter predictions. Based on the adjusted features, the target object's motion is predicted. This helps understand the target object's behavioral patterns, providing useful information for subsequent decision-making and control. Through feature fusion, adjustment, and prediction, accurate understanding and prediction of the target object's motion are achieved, supporting subsequent applications.

[0120] In step 108, the initial feature prediction network is trained based on the predicted action parameters and the label action parameters.

[0121] In some embodiments, the above step 108 can be implemented in the following manner: based on the predicted action parameters and the label action parameters, the initial feature prediction network is trained using reinforcement learning.

[0122] In some embodiments, the predicted action parameters include multiple sub-prediction action parameters, the label action parameters include multiple sub-label action parameters, the sub-prediction action parameters and the sub-label action parameters correspond one-to-one, the sub-prediction action parameters and the corresponding sub-label action parameters belong to the same feature dimension, and the sub-prediction action parameters correspond one-to-one to the feature dimension.

[0123] As an example, the above-mentioned feature dimensions include joint position dimension, joint velocity dimension, root joint posture dimension, and root joint velocity dimension, etc., then the predicted action parameters include sub-prediction action parameters corresponding to the joint position dimension, sub-prediction action parameters corresponding to the joint velocity dimension, sub-prediction action parameters corresponding to the root joint posture dimension, and sub-prediction action parameters corresponding to the root joint velocity dimension. The label action parameters include sub-label action parameters corresponding to the joint position dimension, sub-label action parameters corresponding to the joint velocity dimension, sub-label action parameters corresponding to the root joint posture dimension, and sub-label action parameters corresponding to the root joint velocity dimension.

[0124] In some embodiments, referring to FIG8 , FIG8 is a flowchart diagram five of the robot control method provided in an embodiment of the present application. In step 108 shown in FIG6 , the initial feature prediction network can be trained through steps 1081 to 1083 shown in FIG8 .

[0125] In step 1081, for each feature dimension, the importance weight of the feature dimension is obtained, and based on the corresponding sub-prediction action parameters and sub-label action parameters, the dimension loss value corresponding to the feature dimension is determined.

[0126] As an example, for the joint position dimension, the importance weight of the joint position dimension can be 0.6, and the expression of the dimension loss value corresponding to the joint position dimension can be:

[0127] in, Used to indicate the dimension loss value corresponding to the joint position dimension, Sub-predicted action parameters used to indicate the joint position dimension, Sub-tag action parameters used to indicate joint position dimensions.

[0128] As an example, for the joint velocity dimension, the importance weight of the joint velocity dimension may be 0.05, and the expression of the dimension loss value corresponding to the joint velocity dimension may be:

[0129] in, Used to indicate the dimension loss value corresponding to the joint velocity dimension, Sub-predicted action parameters used to indicate the joint velocity dimension, Sub-tag action parameter used to indicate the joint velocity dimension.

[0130] As an example, for the end effector position dimension, the importance weight of the end effector position dimension may be 0.1, and the expression of the dimension loss value corresponding to the end effector position dimension may be:

[0131] in, Used to indicate the dimension loss value corresponding to the position dimension of the end effector, Sub-predicted action parameters used to indicate the end effector position dimension, Subtag action parameters used to indicate the end effector position dimension.

[0132] As an example, for the root joint posture dimension, the importance weight of the root joint posture dimension can be 0.15, and the expression of the dimension loss value corresponding to the root joint posture dimension can be:

[0133] in, Used to indicate the dimension loss value corresponding to the root joint posture dimension, The sub-predicted action parameters used to indicate the position of the root joint pose dimension, Subtag action parameter used to indicate the position of the root joint pose dimension, Sub-predicted action parameters for indicating the rotation of the root joint pose dimension, Subtag action parameter used to indicate the rotation of the root joint pose dimensions.

[0134] As an example, for the root joint velocity dimension, the importance weight of the root joint velocity dimension can be 0.1, and the expression of the dimension loss value corresponding to the root joint velocity dimension can be:

[0135] in, The dimension loss value used to indicate the dimension of the root joint velocity, Sub-predicted action parameters used to indicate the linear velocity of the root joint velocity dimension, Sub-tag action parameter used to indicate the linear velocity of the root joint velocity dimension. The sub-predicted action parameter used to indicate the angular velocity of the root joint velocity dimension, Subtag action parameter for indicating the angular velocity dimension of the root joint velocity.

[0136] In step 1082, according to the importance weight of each feature dimension, the dimension loss values ​​corresponding to each feature dimension are weighted and summed to obtain the target loss value.

[0137] As an example, the expression for the target loss value can be:

[0138] Among them, r t Used to indicate the target loss value, Used to indicate the dimension loss value corresponding to the joint position dimension, Used to indicate the dimension loss value corresponding to the joint velocity dimension, Used to indicate the dimension loss value corresponding to the root joint posture dimension, A dimensionality penalty value indicating the dimension of the root joint velocity.

[0139] In step 1083, the initial feature prediction network is trained based on the target loss value.

[0140] In some embodiments, the above step 1083 can be implemented as follows: based on the target loss value, the initial feature prediction network is trained by reinforcement learning.

[0141] In other embodiments, the above step 1083 can be implemented as follows: based on the target loss value, the network parameters of the initial feature prediction network are updated by gradient updating.

[0142] By obtaining the importance weights for each feature dimension, we can better understand the contribution of each feature to the task. This helps determine the weights used in feature loss calculations, thereby better utilizing important features. For each feature dimension, a dimension loss value is calculated based on its importance weight and the corresponding sub-prediction action parameters and sub-label action parameters. This facilitates personalized loss calculation for each feature dimension during training, better guiding network optimization. Based on the importance weights of each feature dimension, the dimension loss values ​​for each feature dimension are weighted and summed to obtain the target loss value. This helps factor in the losses of different features and comprehensively assess the overall loss. The initial feature prediction network is trained based on the target loss value. This allows the network to better adapt to task requirements and improve prediction accuracy and robustness. By more carefully considering feature importance and loss, the network training process is optimized, thereby improving model performance and generalization.

[0143] In this way, the initial feature prediction network is trained by the object state data used to indicate the motion state of the target object and the label action parameters of the target object, so that the initial feature prediction network has the performance of imitating the action of the target object. The initial action parameters of the robot are predicted based on the state data and environmental data to obtain the initial action parameters of the robot, so that the obtained initial action parameters can highly imitate the action of the target object, thereby enabling the robot to highly simulate the target object.

[0144] In step 103 , based on the environmental data, environmental impact parameters are predicted for characterizing the impact of the environment on the robot's imitation of the object's motion.

[0145] In some embodiments, the environmental impact parameter is used to indicate the degree of influence of environmental data on the initial action parameters.

[0146] In some embodiments, the environmental impact parameter prediction is obtained through an environmental impact parameter prediction network, which includes a data adjustment layer and an environmental parameter prediction layer.

[0147] In some embodiments, the environmental impact parameter prediction network has the ability to predict environmental impacts, and the environmental impact parameter prediction network can be trained using sample state data so that the trained environmental impact parameter prediction network has the ability to predict environmental impacts.

[0148] As an example, refer to Figure 9, which is a network structure diagram of the environmental impact parameter prediction network provided in an embodiment of the present application. The environmental impact parameter prediction is obtained by the environmental impact parameter prediction network shown in Figure 9. The environmental impact parameter prediction network includes a data adjustment layer 91 and an environmental parameter prediction layer 92.

[0149] In some embodiments, referring to FIG. 10 , FIG. 10 is a flow chart six of the robot control method provided in an embodiment of the present application. Step 103 shown in FIG. 3 can be implemented through steps 1031 to 1032 shown in FIG. 10 .

[0150] In step 1031, the data adjustment layer is called to perform data adjustment on the status data to obtain adjustment reference data.

[0151] As an example, referring to FIG. 9 , the offset data adjustment layer 91 is called to perform data adjustment on the state data b1 to obtain the adjusted reference data b2 .

[0152] In step 1032, the environmental parameter prediction layer is called to predict the robot's motion parameters based on the adjustment reference data and the environmental data to obtain the robot's environmental impact parameters.

[0153] As an example, referring to FIG9 , the environmental parameter prediction layer 92 is called to predict the robot's motion parameters based on the adjustment reference data b2 and the environmental data b3 to obtain the robot's environmental impact parameters atTA.

[0154] In this way, by predicting the environmental impact parameters of the robot based on the adjustment reference data and environmental data, the environmental impact parameters of the robot are obtained, so that the obtained environmental impact parameters can be highly adapted to the environment, thereby enabling the robot to highly adapt to the environment and perform actions.

[0155] In some embodiments, referring to FIG11 , FIG11 is a flow chart seven of the robot control method provided in an embodiment of the present application. Before step 103 shown in FIG3 , the environmental impact parameter prediction network can also be trained by executing steps 109 to 111 as follows.

[0156] In step 109, the sample state data and the tag environment impact parameter are acquired, where the tag environment impact parameter is used to indicate the degree of impact of the environment data on the sample state data.

[0157] In some embodiments, the above-mentioned acquisition of sample state data can be achieved by: acquiring test state information indicating the motion state of the test object collected by a state sensor provided on the test object, performing feature extraction on the test state information, and obtaining sample state data.

[0158] In step 110, an environmental impact parameter prediction network is called to predict environmental impact parameters of a target object based on sample state data to obtain predicted environmental impact parameters.

[0159] In some embodiments, referring to FIG. 12 , FIG. 12 is a flowchart eight of the robot control method provided in an embodiment of the present application. Step 110 shown in FIG. 11 can be implemented through steps 1101 to 1103 shown in FIG. 12 .

[0160] In step 1101, the sample state data and the state data are fused to obtain sample fusion data.

[0161] As an example, referring to FIG9 , the sample state data b4 and the state data b1 are fused to obtain sample fused data b5.

[0162] In step 1102, the data adjustment layer is called to perform data adjustment on the sample fusion data to obtain adjusted sample state data.

[0163] As an example, referring to FIG. 9 , the offset data adjustment layer 91 is called to perform data adjustment on the sample state data b5 to obtain offset-adjusted state data b6 .

[0164] In step 1103, the environmental parameter prediction layer is called to predict the action parameters of the target object based on the adjustment sample state data and the state data to obtain the predicted environmental impact parameters.

[0165] As an example, referring to FIG9 , the environment parameter prediction layer 92 is called to perform action parameter prediction on the target object based on the offset adjustment state data b6 and the state data b1 to obtain the predicted environment impact parameter b7.

[0166] In this way, fusing sample state data with state data can make features richer and more comprehensive, capturing more information. This helps improve the model's understanding and representation of samples. By adjusting the sample state data, offset-adjusted state data is obtained. This helps introduce offset adjustments to the state while considering the sample, making the model more adaptable to changes in different contexts. Based on the adjusted features, the target object's action features are predicted. This helps understand the target object's behavior and make corresponding predictions. By fusing and adjusting features and predicting the target object's action features, accurate understanding and prediction of the target behavior are achieved, providing support for subsequent applications.

[0167] In step 111, an environmental impact parameter prediction network is trained based on the predicted environmental impact parameters and the labeled environmental impact parameters.

[0168] In some embodiments, the above step 111 can be implemented in the following manner: based on the predicted environmental impact parameters and the labeled environmental impact parameters, the environmental impact parameter prediction network is trained using reinforcement learning.

[0169] In other embodiments, the above step 111 may be implemented as follows: based on the predicted environmental impact parameters and the labeled environmental impact parameters, the network parameters of the environmental impact parameter prediction network are trained using a gradient update method.

[0170] In some embodiments, the predicted environmental impact parameters include multiple sub-predicted environmental impact parameters, the label environmental impact parameters include multiple sub-label environmental impact parameters, the sub-predicted environmental impact parameters correspond one-to-one to the sub-label environmental impact parameters, the sub-predicted environmental impact parameters and the corresponding sub-label environmental impact parameters belong to the same feature dimension, and the sub-predicted environmental impact parameters correspond one-to-one to the feature dimension.

[0171] In some embodiments, referring to FIG. 13 , FIG. 13 is a flow chart nine of the robot control method provided in an embodiment of the present application. Step 111 shown in FIG. 11 can be implemented through steps 1111 to 1112 shown in FIG. 13 .

[0172] In step 1111, for each feature dimension, based on the corresponding sub-prediction environment impact parameter and the sub-label environment impact parameter, the dimension loss value corresponding to the feature dimension is determined.

[0173] As an example, for the speed dimension, the expression of the dimension loss value corresponding to the speed dimension can be:

[0174] Among them, L1 is used to indicate the dimensional loss value corresponding to the speed dimension. Sub-prediction environmental impact parameter used to indicate the speed dimension, vt Used to indicate the sub-label environmental impact parameter corresponding to the speed dimension.

[0175] As an example, for the angle dimension, the expression of the dimension loss value corresponding to the angle dimension can be:

[0176] Among them, L2 is used to indicate the dimension loss value corresponding to the angle dimension. The sub-prediction environmental impact parameter used to indicate the angular dimension, θ t Used to indicate the sub-label environmental impact parameter corresponding to the angle dimension.

[0177] In step 1112, a reference loss value is determined based on the loss values ​​of each dimension, and the environmental impact parameter prediction network is trained based on the reference loss value.

[0178] As an example, the expression of the above reference loss value can be:

[0179] Among them, L is used to indicate the reference loss value, and L2 is used to indicate the dimension loss value corresponding to the angle dimension. The sub-prediction environmental impact parameter used to indicate the angular dimension, θ t It is used to indicate the sub-label environmental impact parameter corresponding to the angle dimension, and L1 is used to indicate the dimension loss value corresponding to the speed dimension. Sub-prediction environmental impact parameter used to indicate the speed dimension, v t Used to indicate the sub-label environmental impact parameter corresponding to the speed dimension.

[0180] In this way, the environmental impact parameter prediction network is trained based on the predicted environmental impact parameters and the labeled environmental impact parameters, so that the environmental impact parameter prediction network has the performance of analyzing the impact of environmental data. The environmental impact parameters of the robot are predicted based on the state data and the environmental data, and the environmental impact parameters of the robot are obtained, so that the obtained environmental impact parameters can be highly adapted to the environment, thereby enabling the robot to highly adapt to the environment to perform actions.

[0181] In step 104, the initial motion parameters and the environmental impact parameters are fused to obtain the fused motion parameters of the robot.

[0182] In some embodiments, the fusion refers to a process of fusing at least two different features.

[0183] In some embodiments, referring to FIG. 14 , FIG. 14 is a flowchart diagram 10 of the robot control method provided in an embodiment of the present application. Step 104 shown in FIG. 3 can be implemented through steps 1041 to 1042 shown in FIG. 14 .

[0184] In step 1041 , a fusion coefficient corresponding to the environmental impact parameter is obtained, and the value of the fusion coefficient is positively correlated with the value of the norm of the environmental data.

[0185] In some embodiments, the larger the value of the norm of the environmental data, the larger the value of the corresponding fusion coefficient, and the smaller the value of the norm of the environmental data, the smaller the value of the corresponding fusion coefficient.

[0186] In step 1042, the product of the fusion coefficient and the environmental impact parameter is determined as the target environmental impact parameter, and the target environmental impact parameter is fused with the initial action parameter to obtain the fused action parameter.

[0187] As an example, the expression of the target environmental impact parameter can be: T = αT1 (10)

[0188] Among them, T is used to indicate the target environment impact parameter, α is used to indicate the fusion coefficient, and T1 is used to indicate the environment impact parameter.

[0189] In this way, by fusing the initial action parameters and the environmental influence parameters, the robot's fused action parameters are obtained. Since the obtained environmental influence parameters can be highly adapted to the environment, the robot can highly adapt to the environment to perform actions. The obtained initial action parameters can highly imitate the actions of the target object, so that the robot can highly simulate the target object. The obtained fused action parameters can both be highly adapted to the environment and highly simulate the target object.

[0190] In step 105, a control instruction is generated based on the fused motion parameters.

[0191] In some embodiments, the control instructions are used to control the robot to move according to the target action indicated by the fused action parameters.

[0192] In some embodiments, referring to FIG. 15 , FIG. 15 is a flow chart eleven of the robot control method provided in an embodiment of the present application. Step 105 shown in FIG. 3 can be implemented through steps 1051 to 1053 shown in FIG. 15 .

[0193] In step 1051, mapping relationships between multiple candidate features and corresponding candidate control instructions are obtained, and the fusion action parameters are matched with each candidate feature to obtain the matching degree between the fusion action parameters and each candidate feature.

[0194] As an example, the mapping relationship between multiple candidate features and corresponding candidate control instructions can be: candidate feature A-candidate control instruction A, candidate feature B-candidate control instruction B, candidate feature C-candidate control instruction C, candidate feature D-candidate control instruction D. The fused action parameter is candidate feature C1, and the fused action parameter is matched with each candidate feature to obtain the matching degree of the fused action parameter with each candidate feature. For example, the matching degree between the fused action parameter C1 and candidate feature C is 90%, the matching degree between the fused action parameter C1 and candidate feature A is 20%, the matching degree between the fused action parameter C1 and candidate feature B is 10%, and the matching degree between the fused action parameter C1 and candidate feature D is 15%.

[0195] In step 1052, the candidate feature corresponding to the maximum matching degree is determined as the matching feature that matches the fusion action parameter.

[0196] Continuing with the above example, the candidate feature C corresponding to the maximum matching degree of 90% is determined as the matching feature that matches the fusion action parameter C1.

[0197] In step 1053, based on the mapping relationship, the candidate control instruction corresponding to the matching feature is determined as the control instruction.

[0198] Continuing with the above example, based on the mapping relationships: candidate feature A-candidate control instruction A, candidate feature B-candidate control instruction B, candidate feature C-candidate control instruction C, candidate feature D-candidate control instruction D, candidate control instruction C corresponding to candidate feature C is determined as the control instruction.

[0199] In some embodiments, the above step 105 can also be implemented in the following manner: obtaining an instruction prediction network, calling the instruction prediction network, and predicting the control instruction based on the fused action parameters.

[0200] In some embodiments, when the robot is a legged robot, the number of limbs of the legged robot is a positive integer greater than 1, and the control instructions include sub-control instructions for controlling each limb. The above step 105 can be implemented as follows: for each of the limbs, determine the sub-fusion action parameters corresponding to the limb from the fusion action parameters, and generate the sub-control instructions corresponding to the limb based on the sub-fusion action parameters.

[0201] As an example, when the number of limbs of a legged robot is 2, for limb 1, the sub-fusion action parameter 11 corresponding to limb 1 is determined from the fusion action parameter, and based on the sub-fusion action parameter 11, the sub-control instruction 111 corresponding to limb 1 is generated; for limb 2, the sub-fusion action parameter 21 corresponding to limb 2 is determined from the fusion action parameter, and based on the sub-fusion action parameter 21, the sub-control instruction 211 corresponding to limb 2 is generated.

[0202] In some embodiments, after step 105 above, each limb can be controlled in the following manner: for each limb, based on the sub-control instruction corresponding to the limb, the limb is controlled to move according to the limb action indicated by the corresponding sub-fusion action parameter.

[0203] Continuing with the previous example, for limb 1, based on the sub-control instruction 111 corresponding to limb 1, limb 1 is controlled to move according to the limb action indicated by the corresponding sub-fusion action parameter 11. For limb 2, based on the sub-control instruction 211 corresponding to limb 2, limb 1 is controlled to move according to the limb action indicated by the corresponding sub-fusion action parameter 21.

[0204] In this way, by predicting the robot's initial motion parameters and environmental influence parameters, the initial motion parameters and environmental influence parameters are obtained. By fusing the initial motion parameters and environmental influence parameters, the robot's fused motion parameters are obtained. Since the obtained environmental influence parameters are highly adaptable to the environment, the robot can highly adapt to the environment and perform actions. The obtained initial motion parameters can highly imitate the movements of the target object, allowing the robot to highly simulate the target object. The obtained fused motion parameters are both highly adaptable to the environment and highly simulate the target object. By generating control instructions based on the fused motion parameters, the robot is controlled to move according to the target movement indicated by the fused motion parameters, so that the robot's movements are both highly adaptable to the environment and highly simulate the target object, effectively improving the robot's motion performance.

[0205] Below, an exemplary application of the embodiment of the present application in an actual application scenario of a quadruped robot will be described.

[0206] The embodiment of the present application proposes a general learning framework to control a quadruped robot that can imitate the behavior of real animals and traverse challenging terrains. The method of the embodiment of the present application includes two steps: the first step is imitation learning, which learns from the movements of real animals; the second step is a terrain adaptation step, which enables it to generalize to terrains that have not been seen. The embodiment of the present application uses a motion capture system to capture the movements of Labrador dogs on various terrains to promote terrain-adaptive movement. The experiments of the embodiment of the present application show that the model of the embodiment of the present application can traverse various terrains with natural behavior. The embodiment of the present application deploys the method of the embodiment of the present application on a real quadruped robot, and the speed of climbing stairs reaches 1.1m / s.

[0207] The embodiments of the present application can be applied to the following application scenarios: exploration and rescue: quadruped robots can operate in various harsh and complex environments, such as disaster sites, fires, earthquakes, etc., to provide assistance to rescue workers. Agriculture: quadruped robots can walk in farmland to help farmers complete tasks such as sowing and harvesting. Industrial production: quadruped robots can carry heavy objects in factories and help workers complete repetitive tasks. Medical field: quadruped robots can help people with disabilities walk and provide support, and can also be used for rehabilitation treatment, guiding the blind, etc.

[0208] In some embodiments, referring to Figures 16 and 17, Figure 16 is a flowchart diagram 12 of the robot control method provided in an embodiment of the present application, and Figure 17 is a principle diagram diagram 1 of the robot control method provided in an embodiment of the present application. The robot control method provided in an embodiment of the present application can be implemented through steps 201 to 205 shown in Figures 16 and 17.

[0209] In step 201 , a motion capture system is used to collect movement data of a dog on a terrain.

[0210] In some embodiments, a motion capture system is a technology that uses sensors, cameras, or other devices to record the movement of living beings or objects. In this case, the system might be used to learn about a dog's movements, such as its gait, speed, acceleration, and possibly other relevant information. This data can be used for a variety of purposes, including studying animal behavior, training, and sports biomechanics.

[0211] In step 202 , the motion capture data is retargeted to the robotic dog skeleton.

[0212] In some embodiments, the data collected by the motion capture system (mocapture data) is redirected or mapped onto the skeletal structure of the robotic dog. In this process, by analyzing the movements of the person or animal recorded in the motion capture data, these movement patterns are applied to the skeletal structure of the robotic dog so that the robotic dog can imitate or replicate the movements of the original movement. This process may involve converting information such as joint angles and body posture in the motion capture data into motion instructions suitable for the robotic dog, so that the robotic dog can perform similar movements according to the movement patterns of the person or animal. This technology is commonly used in film and television production, game development, robotics research and other fields.

[0213] In step 203, the quadruped robot imitates the learning step and traces the reference motion.

[0214] In some embodiments, a quadruped robot learns by tracing a reference action. Imitation learning is a type of machine learning in which a robot or intelligent system learns to perform a specific task by observing and imitating the behavior of humans or other entities. In this scenario, the quadruped robot first observes or receives an example of a reference action. This reference action might be a specific movement performed by a human or other animal, such as walking, running, or other movements. The robot then learns how to perform similar actions by tracing back this reference action, i.e., attempting to imitate it.

[0215] In step 204, the terrain adaptation step, the model is trained on more unseen terrains.

[0216] In some embodiments, terrain adaptation refers to the ability of a robot or model to adapt to different or unknown terrain environments and change its behavior or actions accordingly. In this context, the model may refer to a machine learning model or control system that guides the robot's movement or behavior on various terrains. The goal is to train the model on more and different types of terrain to improve its adaptability to unknown terrain. By training on a variety of terrains, the model can learn a wider range of terrain characteristics and conditions and adjust its behavioral strategies or action plans accordingly.

[0217] In step 205, the real machine is tested climbing stairs and can reach a maximum speed of 1.1 m / s.

[0218] An embodiment of the present application uses a motion capture system Vicon to capture motion data from a Labrador and redirects it to the quad robot Max. The method of an embodiment of the present application includes two steps: an imitation learning step and a terrain adaptation step. In the imitation learning step, an embodiment of the present application uses an encoder-decoder architecture to imitate all different reference motion segments. The encoder learns to map a sequence of future reference motion frames to a latent encoding (Latent Embedding) representing the future motion of the target. The decoder learns to map this encoding together with proprioceptive information and external sensory information to joint motor driver commands. In the subsequent terrain adaptation step, an embodiment of the present application fixes the decoder to maintain the naturalness of the motion and trains the strategy in various terrains in an imitation learning manner. This produces a robust strategy that can handle different challenging terrains with natural behavior.

[0219] In some embodiments, the goal of imitation learning is to train a single policy to imitate all reference motion trajectories by optimizing:

[0220] Among them, s represents the robot state information, including joint position, direction, speed, etc. is the state of the reference motion i at time t, and P represents the world dynamics model. In this embodiment, imitation learning is formulated as a reinforcement learning problem. In reinforcement learning, the goal is to learn a policy π to maximize its expected reward.

[0221] At each time step, the agent receives the state s t , and for action a t ~π(a t |s t ) to sample. The agent performs this operation and receives a new state s t+1 and reward r t In the tasks of the embodiment of the present application, the reward is intended to encourage the robot to trace back the reference motion.

[0222] In some embodiments, see Figure 18, which is a second schematic diagram of the principle of the robot control method provided by the embodiment of the present application. Reference Motion Encoder Depends on a set of future reference motions and proprioceptive information in Represents the state of the reference motion in the future 1 / 30, 1 / 15, 1 / 3 and 1 second respectively. RM Learn to map the future reference trajectory into the latent space and generate random commands z t The embodiment of this application will The output of is modeled as a Gaussian distribution with mean The standard deviation is

[0223] In order to ensure the Gaussian distribution of the latent space and perform more focused exploration in the terrain adaptation step, this embodiment of the application regularizes the standard Gaussian distribution by using the KL divergence as a penalty, which is similar to the conditional variational autoencoder (CVAE):

[0224] in is a prior distribution, which is a standard normal distribution in the embodiment of the present application. Then get the hidden variable z t , body status information and external terrain information And generate action a for the robot t The parameters of the underlying controller will remain fixed and will be reused in the next terrain adaptation step.

[0225] Reward function r tConsists of 5 items: Joint position reward Joint Speed ​​Bonus End effector position reward Root Joint Pose Bonus and root joint speed bonus

[0226] in, represents the 1D motion rotation and angular velocity of the robot's joint j at time t, represents the end effector position, is the root joint position and rotation, and represents the root joint linear velocity and angular velocity, represents the reference motion.

[0227] The strategy trained in the imitation learning step is able to trace back the reference motion, but there are still some problems to be solved before it can be deployed in the real world. First, the imitation learning strategy can only trace back the reference motion with a specific terrain from the MoCap dataset of the embodiment of the present application, because the variation of the terrain in the MoCap dataset is limited. The dataset of the embodiment of the present application contains slopes and stairs of fixed sizes, so tracing back such reference motion on terrains of different sizes may result in failure. For example, tracing back the motion of climbing 3 steps when there are actually more steps is likely to hit the stairs. Considering that the cost of collecting a large amount of MoCap data in various terrains is very high, the goal of the embodiment of the present application is to generalize the strategy of the embodiment of the present application to a variety of unseen terrains when the terrain in the dataset is limited.

[0228] In addition, this strategy can only trace back a predefined trajectory without human control. The embodiment of the present application hopes to control the robot under the condition of given target angle and speed.

[0229] To this end, the present embodiment proposes a terrain adaptation step. The goal of this step is to extend the strategy to various terrains under a given command. For this step, the reference motion encoder is replaced by the command encoder. The encoder responds to user commands c t and entity status information Encoding. User command in θ t , They are target angular velocity, actual angular velocity, and target linear velocity. The reward function is:

[0230] In order to promote the effectiveness of the strategy on different terrains, the embodiment of the present application uses a curriculum learning method to train the robot on 7 different terrains at the same time. Since the generated terrain does not necessarily match the terrain in the dataset of the embodiment of the present application, this difference may reduce performance. As a solution, the embodiment of the present application can simultaneously update the underlying controller to adapt to unseen situations and improve performance; however, this may lead to a forgetting problem, that is, the movement becomes unnatural and no longer like a real dog. In order to maintain the naturalness of the movement, the embodiment of the present application fixes the underlying controller during training and proposes a terrain adaptation module π TA , allowing the policy to adapt to new environments while maintaining the original movement style. Terrain Adaptation Module Generate an action offset This offset is added to the output of the underlying controller a t to improve performance on unseen terrains:

[0231] Where α controls the degree of influence of the terrain adaptation module on the action. There is a trade-off between adaptability and naturalness: when α is small, the policy maintains its original movement style but performs poorly on challenging terrain, while when α is large, the robot performs well on the terrain but loses naturalness. The embodiment of the present application observes that the terrain adaptation module is crucial for challenging terrain, while for flat surfaces, the underlying controller can handle it adequately. Therefore, the embodiment of the present application proposes a terrain adaptation parameter α to control the degree of influence of the terrain adaptation module on the action:

[0232] The embodiment of the present application applies the method of the embodiment of the present application to the Max quadruped robot in the real world without any fine-tuning. Although perception is essential for robots, the embodiment of the present application focuses on the imitation and terrain adaptation parts, and therefore uses the Vicon motion capture system to directly obtain external perception information. Specifically, the embodiment of the present application designs a world height map with the same terrain height as in the real world, and then uses Vicon to locate the robot within the map. This enables the embodiment of the present application to collect terrain samples around the robot. All body state information is measured by sensors on the robot. The embodiment of the present application tests the method of the embodiment of the present application on planes, slopes, and stairs. For the stair terrain, the first step is 18 cm high and the rest are 13 cm high. The policy network of the embodiment of the present application learned fast and agile behavior, comparable to animals, with a speed of 1.1 meters per second on the stairs. As far as the embodiment of the present application knows, this is the fastest speed achieved by the Max robot climbing stairs.

[0233] In this way, by predicting the robot's initial motion parameters and environmental influence parameters, the initial motion parameters and environmental influence parameters are obtained. By fusing the initial motion parameters and environmental influence parameters, the robot's fused motion parameters are obtained. Since the obtained environmental influence parameters are highly adaptable to the environment, the robot can highly adapt to the environment and perform actions. The obtained initial motion parameters can highly imitate the movements of the target object, allowing the robot to highly simulate the target object. The obtained fused motion parameters are both highly adaptable to the environment and highly simulate the target object. By generating control instructions based on the fused motion parameters, the robot is controlled to move according to the target movement indicated by the fused motion parameters, so that the robot's movements are both highly adaptable to the environment and highly simulate the target object, effectively improving the robot's motion performance.

[0234] It is understandable that in the embodiments of the present application, when data such as status data and environmental data are involved, when the embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0235] The following continues to describe an exemplary structure of the robot control device 455 provided in an embodiment of the present application implemented as a software module. In some embodiments, as shown in Figure 2, the software modules stored in the robot control device 455 of the memory 450 may include: a feature acquisition module 4551, configured to obtain state data indicating the current motion state of the robot, and obtain environmental data indicating the current environment of the robot; an initial action prediction module 4552, configured to predict initial action parameters for controlling the robot to imitate the object action of the target object based on the state data; an offset action prediction module 4553, configured to predict environmental influence parameters for characterizing the influence of the environment on the robot's imitation of the object action based on the environmental data; a fusion module 4554, configured to fuse the initial action parameters and the environmental influence parameters to obtain the fused action parameters of the robot; a generation module 4555, configured to generate control instructions based on the fused action parameters, and the control instructions are used to control the robot to perform the target action indicated by the fused action parameters.

[0236] In some embodiments, the above-mentioned initial action parameters are predicted by an initial feature prediction network, and the initial feature prediction network includes an initial data adjustment layer and an initial parameter prediction layer. The above-mentioned initial action prediction module is also configured to call the initial data adjustment layer to perform data adjustment on the state data to obtain initial reference data; call the initial parameter prediction layer to predict the action parameters of the robot based on the initial reference data and the environmental data to obtain the initial action parameters of the robot.

[0237] In some embodiments, the above-mentioned initial action prediction module is further configured to obtain object state data indicating the motion state of the target object and label action parameters of the target object; call the initial feature prediction network, and predict the action parameters of the target object based on the object state data to obtain the predicted action parameters of the target object; and train the initial feature prediction network based on the predicted action parameters and the label action parameters.

[0238] In some embodiments, the above-mentioned initial action prediction module is further configured to fuse the object state data and the state data to obtain state fusion data; call the initial data adjustment layer to perform data adjustment on the state fusion data to obtain initial adjusted state data; call the initial parameter prediction layer to predict the action parameters of the target object based on the initial adjusted state data and the state data to obtain the predicted action parameters of the target object.

[0239] In some embodiments, the predicted action parameters include multiple sub-prediction action parameters, the label action parameters include multiple sub-label action parameters, the sub-prediction action parameters and the sub-label action parameters correspond one-to-one, the sub-prediction action parameters and the corresponding sub-label action parameters belong to the same feature dimension, and the sub-prediction action parameters correspond one-to-one to the feature dimension; the above-mentioned initial action prediction module is also configured to obtain the importance weight of the feature dimension for each feature dimension, and determine the dimension loss value corresponding to the feature dimension based on the corresponding sub-prediction action parameters and the sub-label action parameters; according to the importance weight of each feature dimension, the dimension loss value corresponding to each feature dimension is weighted and summed to obtain the target loss value; based on the target loss value, the initial feature prediction network is trained.

[0240] In some embodiments, the environmental impact parameter prediction is obtained through an environmental impact parameter prediction network, and the environmental impact parameter prediction network includes a data adjustment layer and an environmental parameter prediction layer. The above-mentioned initial action prediction module is also configured to call the data adjustment layer to perform data adjustment on the state data to obtain adjustment reference data; call the environmental parameter prediction layer, and based on the adjustment reference data and the environmental data, the robot performs action parameter prediction to obtain the environmental impact parameters of the robot.

[0241] In some embodiments, the above-mentioned initial action prediction module is further configured to obtain sample state data and label environmental impact parameters, wherein the label environmental impact parameters are used to indicate the degree of influence of the environmental data on the sample state data; call the environmental impact parameter prediction network, and predict the environmental impact parameters of the target object based on the sample state data to obtain predicted environmental impact parameters; and train the environmental impact parameter prediction network based on the predicted environmental impact parameters and the label environmental impact parameters.

[0242] In some embodiments, the above-mentioned initial action prediction module is further configured to fuse the sample state data and the state data to obtain sample fusion data; call the data adjustment layer to perform data adjustment on the sample fusion data to obtain adjusted sample state data; call the environmental parameter prediction layer to perform action parameter prediction on the target object based on the adjusted sample state data and the state data to obtain the predicted environmental impact parameters.

[0243] In some embodiments, the predicted environmental impact parameters include multiple sub-predicted environmental impact parameters, the label environmental impact parameters include multiple sub-label environmental impact parameters, the sub-predicted environmental impact parameters correspond one-to-one to the sub-label environmental impact parameters, the sub-predicted environmental impact parameters and the corresponding sub-label environmental impact parameters belong to the same feature dimension, and the sub-predicted environmental impact parameters correspond one-to-one to the feature dimension; the above-mentioned offset action prediction module is also configured to determine, for each of the feature dimensions, the dimension loss value corresponding to the feature dimension based on the corresponding sub-predicted environmental impact parameter and the sub-label environmental impact parameter; determine a reference loss value based on each of the dimension loss values, and train the environmental impact parameter prediction network based on the reference loss value.

[0244] In some embodiments, the above-mentioned fusion module is further configured to obtain a fusion coefficient corresponding to the environmental impact parameter, and the value of the fusion coefficient is positively correlated with the value of the norm of the environmental data; the product of the fusion coefficient and the environmental impact parameter is determined as the target environmental impact parameter, and the target environmental impact parameter and the initial action parameter are fused to obtain the fused action parameter.

[0245] In some embodiments, the above-mentioned generation module is further configured to obtain the mapping relationship between multiple candidate features and corresponding candidate control instructions, and match the fusion action parameters with each candidate feature respectively to obtain the matching degree between the fusion action parameters and each candidate feature; the candidate feature corresponding to the largest matching degree is determined as the matching feature that matches the fusion action parameter; based on the mapping relationship, the candidate control instruction corresponding to the matching feature is determined as the control instruction.

[0246] In some embodiments, the above-mentioned feature acquisition module is also configured to obtain status information collected by the status sensor set on the robot, and the status information is used to indicate the current motion state of the robot; perform feature extraction on the status information to obtain the status data; obtain environmental information collected by the environmental sensor set on the robot, and the environmental information is used to indicate the current environment of the robot; perform feature extraction on the environmental information to obtain the environmental data.

[0247] In some embodiments, when the robot is a legged robot, the number of limbs of the legged robot is a positive integer greater than 1, and the control instructions include sub-control instructions for controlling each of the limbs; the above-mentioned generation module is also configured to determine the sub-fusion action parameters corresponding to the limb from the fusion action parameters for each of the limbs, and generate the sub-control instructions corresponding to the limb based on the sub-fusion action parameters; the above-mentioned generation module is also configured to control the limb to move according to the limb action indicated by the corresponding sub-fusion action parameters based on the sub-control instructions corresponding to the limb.

[0248] An embodiment of the present application provides a computer program product, which includes a computer program or computer-executable instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the robot control method described above in the embodiment of the present application.

[0249] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions, wherein computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, the processor will execute the robot control method provided by an embodiment of the present application, for example, the robot control method shown in Figure 3.

[0250] In some embodiments, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), an erasable programmable read-only memory (EPROM), a flash memory, a magnetic surface memory, an optical disk, or a CD-ROM; or it may be various electronic devices including one or any combination of the above memories.

[0251] In some embodiments, computer-executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0252] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file storing other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file specifically configured for the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0253] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0254] By way of example, computer-executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed across multiple sites and interconnected by a communication network.

[0255] In summary, the embodiments of the present application have the following beneficial effects:

[0256] (1) The initial motion parameters and environmental influence parameters of the robot are predicted, and the initial motion parameters and environmental influence parameters are obtained. The initial motion parameters and environmental influence parameters are fused to obtain the robot's fused motion parameters. Since the obtained environmental influence parameters are highly adaptable to the environment, the robot can highly adapt to the environment and perform actions. The obtained initial motion parameters can highly imitate the actions of the target object, so that the robot can highly simulate the target object. The obtained fused motion parameters can both highly adapt to the environment and highly simulate the target object. The control instructions generated based on the fused motion parameters are used to control the robot to move according to the target action indicated by the fused motion parameters, so that the actions performed by the robot can both highly adapt to the environment and highly simulate the target object, which can effectively improve the robot's motion performance.

[0257] (2) The initial motion parameters of the robot are predicted based on the state data and the environment data, and the initial motion parameters of the robot are obtained so that the obtained initial motion parameters can highly imitate the motion of the target object, thereby enabling the robot to highly simulate the target object.

[0258] (3) The initial feature prediction network is trained using the object state data indicating the motion state of the target object and the label motion parameters of the target object, so that the initial feature prediction network has the performance of imitating the motion of the target object. The initial motion parameters of the robot are predicted based on the state data and the environmental data to obtain the initial motion parameters of the robot, so that the obtained initial motion parameters can highly imitate the motion of the target object, thereby enabling the robot to highly simulate the target object.

[0259] (4) By predicting the environmental impact parameters of the robot based on the adjusted reference data and environmental data, the environmental impact parameters of the robot are obtained, so that the obtained environmental impact parameters can be highly adapted to the environment, thereby enabling the robot to highly adapt to the environment and perform actions.

[0260] (5) By training the environmental impact parameter prediction network based on the predicted environmental impact parameters and the labeled environmental impact parameters, the environmental impact parameter prediction network has the performance of analyzing the impact of environmental data. By predicting the environmental impact parameters of the robot based on the state data and the environmental data, the environmental impact parameters of the robot are obtained, so that the obtained environmental impact parameters can be highly adapted to the environment, thereby enabling the robot to highly adapt to the environment and perform actions.

[0261] (6) By fusing the initial action parameters and the environmental influence parameters, the robot's fused action parameters are obtained. Since the obtained environmental influence parameters can be highly adapted to the environment, the robot can highly adapt to the environment and perform actions. The obtained initial action parameters can highly imitate the actions of the target object, so that the robot can highly simulate the target object. The obtained fused action parameters can both be highly adapted to the environment and highly simulate the target object.

[0262] (7) By fusing object state data and status data, different types of information can be integrated together, enhancing the richness and expressiveness of features. This helps improve the performance and accuracy of subsequent steps. By adjusting the initial features to make them more suitable for the target task or scenario, this helps resolve mismatches or deviations between features, thereby improving the accuracy of subsequent action parameter predictions. Based on the adjusted features, the action of the target object is predicted. This can help understand the behavior pattern of the target object, thereby providing useful information for subsequent decision-making and control. By fusing, adjusting, and predicting features, accurate understanding and prediction of the target object's actions are achieved, providing support for subsequent applications.

[0263] (8) By obtaining the importance weights of each feature dimension, we can better understand the contribution of each feature to the task. This helps to determine the weights in the feature loss calculation, so as to better utilize important features. For each feature dimension, the dimension loss value is calculated based on its importance weight and the corresponding sub-prediction action parameters and sub-label action parameters. This helps to perform personalized loss calculation for each feature dimension during training to better guide network optimization. According to the importance weights of each feature dimension, the dimension loss values ​​of each feature dimension are weighted and summed to obtain the target loss value. This helps to take into account the losses of different features and evaluate the overall loss situation. Based on the target loss value, the initial feature prediction network is trained. This can make the network better adapt to the task requirements and improve the accuracy and robustness of the prediction. By more carefully considering the importance and loss of features, the network training process is optimized, thereby improving the performance and generalization ability of the model.

[0264] (9) Fusion of sample state data and state data can make features richer and more comprehensive, capturing more information. This helps improve the model's ability to understand and express samples. By adjusting the sample state data, offset-adjusted state data is obtained. This helps introduce offset adjustments to the state while considering the sample, making the model more adaptable to changes in different situations. Based on the adjusted features, the action features of the target object are predicted. This can help understand the behavior of the target object and make corresponding predictions. By fusing and adjusting features, and predicting the action features of the target object, accurate understanding and prediction of the target behavior can be achieved, providing support for subsequent applications.

[0265] The above description is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, and improvements made within the spirit and scope of the present application are included in the scope of protection of the present application.

Claims

1. A robot control method, the method comprising: Acquire state data indicating a current motion state of the robot, and acquire environment data indicating a current environment of the robot; Based on the state data, predicting initial motion parameters for controlling the robot to imitate an object motion of a target object; Based on the environmental data, predicting environmental impact parameters for characterizing the impact of the environment on the robot's imitation of the object's action; Fusing the initial motion parameters and the environmental impact parameters to obtain fused motion parameters of the robot; Based on the fused action parameters, a control instruction is generated, where the control instruction is used to control the robot to execute a target action indicated by the fused action parameters.

2. The method according to claim 1, wherein: The initial action parameters are predicted by an initial feature prediction network, the initial feature prediction network includes an initial data adjustment layer and an initial parameter prediction layer, and the initial action parameters used to control the robot to imitate the object action of the target object based on the state data are predicted, including: Calling the initial data adjustment layer to perform data adjustment on the state data to obtain initial reference data; The initial parameter prediction layer is called to predict the motion parameters of the robot based on the initial reference data and the environmental data to obtain the initial motion parameters of the robot.

3. The method according to claim 2, wherein: Before predicting the initial motion parameters for controlling the robot to imitate the object motion of the target object based on the state data, the method further includes: Acquire object state data indicating the motion state of the target object and tag action parameters of the target object; Calling the initial feature prediction network, and predicting the action parameters of the target object based on the object state data to obtain the predicted action parameters of the target object; The initial feature prediction network is trained based on the prediction action parameters and the label action parameters.

4. The method according to claim 3, wherein: The calling of the initial feature prediction network to predict the action parameters of the target object based on the object state data to obtain the predicted action parameters of the target object includes: Fusing the object state data with the state data to obtain state fusion data; Calling the initial data adjustment layer to perform data adjustment on the state fusion data to obtain initial adjusted state data; The initial parameter prediction layer is called to predict the action parameters of the target object based on the initial adjustment state data and the state data to obtain the predicted action parameters of the target object.

5. The method according to any one of claims 3 to 4, wherein: The prediction action parameter includes a plurality of sub-prediction action parameters, the label action parameter includes a plurality of sub-label action parameters, the sub-prediction action parameters correspond to the sub-label action parameters one-to-one, the sub-prediction action parameters and the corresponding sub-label action parameters belong to the same feature dimension, and the sub-prediction action parameters correspond to the feature dimension one-to-one; The training of the initial feature prediction network based on the prediction action parameter and the label action parameter includes: For each of the feature dimensions, obtain the importance weight of the feature dimension, and determine the dimension loss value corresponding to the feature dimension based on the corresponding sub-prediction action parameter and the sub-label action parameter; According to the importance weight of each feature dimension, the dimension loss values ​​corresponding to each feature dimension are weighted and summed to obtain a target loss value; Based on the target loss value, the initial feature prediction network is trained.

6. The method according to any one of claims 1 to 5, wherein: The environmental impact parameter is predicted by an environmental impact parameter prediction network, the environmental impact parameter prediction network includes a data adjustment layer and an environmental parameter prediction layer, and the environmental impact parameter used to characterize the impact of the environment on the robot's imitation of the object action based on the environmental data includes: Calling the data adjustment layer to perform data adjustment on the status data to obtain adjustment reference data; The environmental parameter prediction layer is called, and based on the adjustment reference data and the environmental data, the robot performs environmental impact parameter prediction to obtain the environmental impact parameters of the robot.

7. The method according to any one of claims 1 to 6, wherein: Before predicting, based on the environmental data, an environmental impact parameter used to characterize the impact of the environment on the robot's imitation of the object's action, the method further includes: Acquire sample state data and a tag environment impact parameter, wherein the tag environment impact parameter is used to indicate the degree of impact of the environment data on the sample state data; Calling the environmental impact parameter prediction network to predict the environmental impact parameters of the target object based on the sample state data to obtain predicted environmental impact parameters; The environmental impact parameter prediction network is trained based on the predicted environmental impact parameters and the label environmental impact parameters.

8. The method according to any one of claims 1 to 7, wherein: The calling of the environmental impact parameter prediction network to predict the environmental impact parameters of the target object based on the sample state data to obtain the predicted environmental impact parameters includes: Fusing the sample state data with the state data to obtain sample fusion data; Calling the data adjustment layer to perform data adjustment on the sample fusion data to obtain adjusted sample state data; The environmental parameter prediction layer is called to predict the action parameters of the target object based on the adjustment sample state data and the state data to obtain the predicted environmental impact parameters.

9. The method according to any one of claims 1 to 8, wherein: The predicted environmental impact parameter includes a plurality of sub-predicted environmental impact parameters, the label environmental impact parameter includes a plurality of sub-label environmental impact parameters, the sub-predicted environmental impact parameters correspond one-to-one with the sub-label environmental impact parameters, the sub-predicted environmental impact parameters and the corresponding sub-label environmental impact parameters belong to the same feature dimension, and the sub-predicted environmental impact parameters correspond one-to-one with the feature dimension; The step of training the environmental impact parameter prediction network based on the predicted environmental impact parameter and the label environmental impact parameter includes: For each of the feature dimensions, based on the corresponding sub-prediction environment impact parameter and the sub-label environment impact parameter, determine a dimension loss value corresponding to the feature dimension; Based on the loss values ​​of each dimension, a reference loss value is determined, and based on the reference loss value, the environmental impact parameter prediction network is trained.

10. The method according to any one of claims 1 to 9, wherein: The fusing the initial motion parameters and the environmental impact parameters to obtain the fused motion parameters of the robot includes: Acquire a fusion coefficient corresponding to the environmental impact parameter, wherein a value of the fusion coefficient is positively correlated with a value of a norm of the environmental data; The product of the fusion coefficient and the environmental impact parameter is determined as the target environmental impact parameter, and the target environmental impact parameter is fused with the initial action parameter to obtain the fused action parameter.

11. The method according to any one of claims 1 to 10, wherein: The generating a control instruction based on the fusion action parameter includes: Acquire mapping relationships between a plurality of candidate features and corresponding candidate control instructions, and match the fused action parameters with each of the candidate features to obtain matching degrees between the fused action parameters and each of the candidate features; Determine the candidate feature corresponding to the largest matching degree as the matching feature matching the fusion action parameter; Based on the mapping relationship, the candidate control instruction corresponding to the matching feature is determined as the control instruction.

12. The method according to any one of claims 1 to 11, wherein: The obtaining of state data indicating the current motion state of the robot includes: Acquire state information collected by a state sensor provided on the robot, wherein the state information is used to indicate a current motion state of the robot; Extracting features from the state information to obtain the state data; The acquiring of environmental data indicating the current environment of the robot comprises: Acquire environmental information collected by an environmental sensor provided on the robot, wherein the environmental information is used to indicate the current environment of the robot; Feature extraction is performed on the environmental information to obtain the environmental data.

13. The method according to any one of claims 1 to 12, wherein: When the robot is a legged robot, the number of legs of the legged robot is a positive integer greater than 1, and the control instruction includes a sub-control instruction for controlling each leg; The generating a control instruction based on the fusion action parameter includes: For each of the foot limbs, determining the sub-fusion action parameters corresponding to the foot limb from the fusion action parameters, and generating the sub-control instructions corresponding to the foot limb based on the sub-fusion action parameters; After generating the control instruction based on the fusion action parameter, the method further includes: For each of the limbs, based on the sub-control instructions corresponding to the limb, the limb is controlled to perform limb movements as indicated by the corresponding sub-fusion action parameters.

14. A robot control device, comprising: A feature acquisition module configured to acquire state data indicating a current motion state of the robot and acquire environment data indicating a current environment of the robot; an initial motion prediction module configured to predict initial motion parameters for controlling the robot to imitate an object motion of a target object based on the state data; an offset action prediction module, configured to predict, based on the environmental data, an environmental impact parameter used to characterize the impact of the environment on the robot's imitation of the object's action; a fusion module configured to fuse the initial motion parameters and the environmental impact parameters to obtain fused motion parameters of the robot; A generation module is configured to generate a control instruction based on the fusion action parameter, wherein the control instruction is used to control the robot to perform a target action indicated by the fusion action parameter.

15. An electronic device, comprising: a memory configured to store computer executable instructions or computer programs; The processor is configured to implement the robot control method according to any one of claims 1 to 13 when executing the computer executable instructions or computer programs stored in the memory.

16. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the robot control method according to any one of claims 1 to 13.

17. A computer program product, comprising a computer program or a computer executable instruction, wherein the computer program or the computer executable instruction, when executed by a processor, implements the robot control method according to any one of claims 1 to 13.

Images