A robot control method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application disclose a robot control method and device, electronic equipment and storage medium, and relate to the technical field of robots and artificial intelligence, wherein the method comprises: obtaining a target task of a robot and a corresponding performance index, obtaining state data, action data and a reward function according to the target task of the robot and the corresponding performance index, constructing a virtual environment through a physical engine, designing a structure of the robot according to the target task of the robot and the corresponding performance index, obtaining state parameters of the robot, obtaining a control instruction according to the state parameters of the robot and environment state information of the virtual environment through a Transformer network, a reinforcement learning algorithm and a diagonal Gaussian policy, and controlling movement of the robot through the control instruction. The present application realizes efficient and accurate control of the robot, solves the problems of low efficiency, poor intelligence level and poor universality in the prior art, and is difficult to combine with the structure information of the robot to make a response.

Description

Technical Field

[0001] This invention relates to the fields of robotics and artificial intelligence, and in particular to a robot control method, device, electronic device, and storage medium. Background Technology

[0002] Currently, in the field of robot control theory, despite the significant progress made in artificial intelligence (AI) technology, there are still significant shortcomings in the universality and cutting-edge applications of robot control algorithms.

[0003] First, most existing robot control algorithms remain at a relatively basic technical level, such as multilayer perceptrons (MLPs) and graph neural networks. These algorithms are inadequate when handling complex tasks, especially in effectively capturing the robot's fine structural information and its dynamic interactions with the environment. Furthermore, existing algorithms lack versatility and cannot be transferred between different types of robots. This limitation restricts the effectiveness of robots in complex and changing environments, failing to meet the growing demand for intelligent robots.

[0004] Secondly, although the Transformer model, as a cutting-edge achievement in AI generation, demonstrates powerful sequence modeling capabilities, its application in robot control and decision-making faces numerous challenges. On the one hand, the Transformer model typically employs an end-to-end token prediction approach. This mechanism often leads to low execution efficiency when handling real-time robot control tasks, making it difficult to meet the demands for rapid response and precise control. On the other hand, the application of the Transformer model in embodied intelligence algorithms relies on dynamic formulas related to the robot's structure, resulting in poor algorithm transferability. This means that whenever faced with robots with different structures or dynamic characteristics, a significant amount of research and development work or training processes need to be carried out again. This not only increases development costs but also prolongs product launch time, limiting the widespread application of the technology. It is evident that existing robot control algorithms have significant shortcomings in terms of versatility, intelligence level, and execution efficiency, making it difficult to fully realize the potential of AI technology.

[0005] Therefore, there is a need for a robot control method that can efficiently integrate AI technology, and is efficient, flexible and highly transferable. Summary of the Invention

[0006] The present invention provides a robot control method to address the problems of low efficiency, poor intelligence and versatility, and difficulty in responding based on the robot's own structural information in existing technologies. The technical solution is as follows:

[0007] According to one aspect of the present invention, a robot control method is provided, the method comprising: acquiring a target task of a robot and corresponding performance indicators; the target task includes path planning, object grasping, and obstacle avoidance; the performance indicators include accuracy, speed, and stability; obtaining state data, motion data, and a reward function based on the target task and corresponding performance indicators of the robot, and constructing a virtual environment through a physics engine; designing the structure of the robot based on the target task and corresponding performance indicators of the robot, and acquiring state parameters of the robot; the structure includes links and joints; obtaining control commands based on the state parameters of the robot and the environmental state information of the virtual environment through a Transformer network, a reinforcement learning algorithm, and a diagonal Gaussian policy, and controlling the movement of the robot through the control commands.

[0008] In one embodiment, obtaining state data, motion data, and a reward function based on the robot's target task and corresponding performance indicators is achieved through the following steps: determining key variables, control commands, and a reward function based on the robot's target task and corresponding performance indicators; the key variables include position, velocity, acceleration, and attitude; the control commands include joint rotation angle, velocity, and torque; organizing the key variables into a state vector as state data, and encoding the control commands into motion vectors as motion data.

[0009] In one embodiment, the virtual environment is constructed using a physics engine through the following steps: determining the physics engine and configuring its parameters based on the robot's target task and corresponding performance indicators; constructing the robot's virtual environment within the physics engine; the virtual environment including terrain, obstacles, and target objects; setting the robot's initial position, posture, and motion state within the virtual environment; and initializing the environmental parameters and sensor data within the virtual environment.

[0010] In one embodiment, the design of the robot's structure based on the robot's target task and corresponding performance indicators is achieved through the following steps: designing the robot's structure and structural parameters based on the robot's target task and corresponding performance indicators; the structural parameters include the number, size, and layout of the structure; converting the designed robot into a file format compatible with a third-party simulator and importing it into the physics engine for testing; and adjusting the robot's structure and structural parameters based on the test results until the test is passed.

[0011] In one embodiment, the control commands obtained based on the robot's state parameters and the environmental state information of the virtual environment through a Transformer network, reinforcement learning algorithm, and diagonal Gaussian policy are achieved through the following steps: inputting the state parameters and environmental state information into the Transformer network to obtain the mean and logarithmic standard deviation of the diagonal Gaussian distribution; obtaining control commands based on the mean and logarithmic standard deviation of the diagonal Gaussian distribution, controlling the robot to execute the control commands to obtain motion results and reward information; and optimizing the parameters of the Transformer network based on the motion results and reward information using a reinforcement learning algorithm.

[0012] In one embodiment, the method further includes the following steps: generating motion commands through the updated Transformer network to control the robot to move in the virtual environment and obtain test results; adjusting the parameters of the Transformer network, the parameters of the reinforcement learning algorithm, the structure and structural parameters of the robot according to the test results until the test results meet the set conditions.

[0013] In one embodiment, controlling the robot to execute the control command to obtain a motion result is achieved through the following steps: calculating the minimum distance between two components in the robot on the structure based on the motion result, and obtaining the motion result based on the minimum distance; the minimum distance is used to evaluate whether the Transformer network can capture the robot structure.

[0014] According to one aspect of the present invention, a robot control device includes: a control target acquisition module for acquiring a robot's target task and corresponding performance indicators; a virtual environment construction module for obtaining state data, motion data, and a reward function based on the robot's target task and corresponding performance indicators, and constructing a virtual environment using a physics engine; a robot structure design module for designing the robot's structure based on the robot's target task and corresponding performance indicators, and acquiring the robot's state parameters; and a control command generation module for obtaining control commands based on the robot's state parameters and the environmental state information of the virtual environment using a Transformer network, a reinforcement learning algorithm, and a diagonal Gaussian policy, and controlling the robot's movement using the control commands.

[0015] According to one aspect of the present invention, an electronic device includes at least one processor and at least one memory, wherein computer-readable instructions are stored on the memory; the computer-readable instructions are executed by one or more of the processors to cause the electronic device to implement the robot control method as described above.

[0016] According to one aspect of the invention, a storage medium stores computer-readable instructions thereon, which are executed by one or more processors to implement the robot control method described above.

[0017] The beneficial effects of the technical solution provided by this invention are:

[0018] In the above technical solution, the present invention first obtains the robot's target task and corresponding performance indicators, then obtains state data, motion data, and a reward function based on the robot's target task and corresponding performance indicators, constructs a virtual environment through a physics engine, designs the robot's structure based on the robot's target task and corresponding performance indicators, obtains the robot's state parameters, and obtains control commands based on the robot's state parameters and the environmental state information of the virtual environment through a Transformer network, reinforcement learning algorithm, and diagonal Gaussian policy. The robot's movement is controlled by the control commands. By combining the Transformer network, reinforcement learning algorithm, and diagonal Gaussian policy to generate control commands, efficient and precise control of the robot is achieved, accelerating the algorithm iteration and robot performance optimization process. This effectively solves the problems of low efficiency, poor intelligence and versatility, and difficulty in responding based on the robot's own structural information in existing technologies. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating a robot control method according to an exemplary embodiment;

[0021] Figure 2 yes Figure 1 The flowchart for constructing the virtual environment in step 130 of the corresponding embodiment;

[0022] Figure 3 yes Figure 1 The flowchart of step 170 in the corresponding embodiment;

[0023] Figure 4 This is a flowchart illustrating a robot control method in an application scenario;

[0024] Figure 5 yes Figure 4 A flowchart illustrating the process of generating joint motion vectors in the corresponding application scenario;

[0025] Figure 6 This is a block diagram illustrating a robot control device according to an exemplary embodiment;

[0026] Figure 7 This is a hardware structure diagram of an electronic device according to an exemplary embodiment;

[0027] Figure 8 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0028] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0029] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this disclosure means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0030] This invention provides a robot control method that generates control commands by combining Transformer networks, reinforcement learning algorithms, and diagonal Gaussian policies. This achieves efficient and precise robot control, solving the problems of low efficiency, poor intelligence and versatility, and difficulty in responding to robot structural information in existing technologies. This robot control method is applicable to robot control devices, which can be electronic devices or run on an external host via control software; no limitation is made here. The robot control method in this invention can be applied to various scenarios, such as robot control.

[0031] Please see Figure 1 This invention provides a robot control method applicable to electronic devices.

[0032] In the following method embodiments, for ease of description, the execution subject of each step of the method is an electronic device, but this does not constitute a specific limitation.

[0033] like Figure 1 As shown, the method may include the following steps:

[0034] Step 110: Obtain the robot's target task and corresponding performance indicators.

[0035] The target tasks include path planning, object grasping, and obstacle avoidance, while the performance indicators include accuracy, speed, and stability, none of which are specified here.

[0036] Step 130: Obtain state data, motion data, and reward function based on the robot's target task and corresponding performance indicators, and construct a virtual environment using a physics engine.

[0037] In one possible implementation, key variables, control commands, and reward functions are determined based on the robot's target task and corresponding performance indicators. The key variables are organized into state vectors as state data, and the control commands are encoded into action vectors as action data.

[0038] Key variables include position, velocity, acceleration, attitude, etc., and control commands include joint rotation angle, velocity, torque, etc., none of which are specified here.

[0039] In one possible implementation, such as Figure 2 As shown, building a virtual environment using a physics engine can include the following steps:

[0040] Step 210: Determine the physics engine and configure its parameters based on the robot's target task and corresponding performance indicators.

[0041] Specifically, the most suitable physics engine is selected to simulate the robot. The choice of physics engine depends on whether it can accurately reflect the physical interaction between the robot and the environment, such as gravity, friction, and collision. After selecting the physics engine, the engine parameters, such as time step and physical accuracy, are adjusted according to the specific characteristics of the robot and the simulation requirements to ensure the accuracy and efficiency of the simulation.

[0042] The physics engine can include I SAAC SIM, Mujoco, pymunk, etc., without any specific limitation.

[0043] Step 230: Build the robot's virtual environment in the physics engine.

[0044] The virtual environment includes terrain, obstacles, and target objects, etc., without being limited here.

[0045] Specifically, the environment needs to simulate the real-world scenarios in which the robot will operate as closely as possible. Environment construction includes adding terrain (such as flat ground, slopes, stairs, etc.), obstacles (such as walls, furniture, other robots, etc.), and target objects (such as objects to be grasped, locations to be navigated to, etc.). These elements need to be designed and placed according to actual needs to ensure that the simulation can realistically reflect the various situations that the robot may encounter in the real world.

[0046] Step 250: Set the robot's initial position, posture, and motion state in the virtual environment; initialize the environmental parameters and sensor data in the virtual environment.

[0047] Step 150: Design the robot's structure based on the robot's target task and corresponding performance indicators, and obtain the robot's state parameters.

[0048] The structure of a robot includes several links and joints, which can form a tree structure, with links being nodes and joints being edges. Links are ordinary rigid bodies with physical properties such as mass and volume. Joints represent the motion constraints between links and have no inherent physical properties such as mass and volume, and will not collide, such as in a robot with an URDF structure.

[0049] In one possible implementation, the robot's structure and structural parameters are designed based on the robot's target task and corresponding performance indicators. The designed robot is then converted into a file format that matches a third-party simulator and imported into a physics engine for testing. The robot's structure and structural parameters are adjusted based on the test results until the test is passed.

[0050] The structural parameters include the number, size, and layout of the structures, etc., which are not specified here.

[0051] Specifically, the parameters and data need to reflect the environmental conditions and perception information that the robot may encounter in actual operation, in order to ensure the authenticity and reliability of the simulation results.

[0052] Step 170: Control commands are obtained based on the robot's state parameters and the environmental state information of the virtual environment through Transformer network, reinforcement learning algorithm and diagonal Gaussian policy, and the robot's movement is controlled by the control commands.

[0053] In one possible implementation, such as Figure 3 As shown, step 170 may include the following steps:

[0054] Step 310: Input the state parameters and environmental state information into the Transformer network to obtain the mean and logarithmic standard deviation of the diagonal Gaussian distribution.

[0055] Specifically, the Transformer network processes this information through a complex self-attention mechanism, outputting the mean and logarithmic standard deviation of the diagonal Gaussian distribution of the control policy, providing a probability distribution basis for the generation of control commands.

[0056] Step 330: Based on the mean and logarithmic standard deviation of the diagonal Gaussian distribution, control instructions are obtained to control the robot to execute the control instructions and obtain motion results and reward information.

[0057] Specifically, based on the mean and logarithmic standard deviation of the diagonal Gaussian distribution output by the Transformer network, specific control commands are obtained by sampling. These control commands include parameters such as the robot's motion direction and speed, which are used to guide the robot's actual motion in the virtual environment. After executing the control commands, the robot's motion results (such as final position, posture changes, etc.) and the resulting reward information (such as whether the task was successfully completed, whether a collision occurred, etc.) are observed and recorded.

[0058] Step 350: Optimize the parameters of the Transformer network based on the motion results and reward information using a reinforcement learning algorithm.

[0059] Specifically, reinforcement learning algorithms (such as Soft Actor-Critic) are used to optimize the parameters of the Transformer network based on the robot's motion results and reward information in the virtual environment. By continuously adjusting the network parameters, the robot can obtain higher cumulative rewards when performing the same task, thereby improving the robot's autonomous decision-making and motion control capabilities in complex environments. This process is an iterative cycle, gradually approaching the optimal control strategy through continuous trial and error and learning.

[0060] In one possible implementation, the updated Transformer network generates motion commands to control the robot's movement in a virtual environment to obtain test results. Based on the test results, the parameters of the Transformer network, the parameters of the reinforcement learning algorithm, the robot's structure, and structural parameters are adjusted until the test results meet the set conditions.

[0061] In one possible implementation, the minimum distance between two components in the robot on the structure is calculated based on the motion results, and the motion results are obtained based on the minimum distance. The minimum distance is used to evaluate whether the Transformer network can capture the robot structure.

[0062] Through the above process, this invention first obtains the robot's target task and corresponding performance indicators, then obtains state data, motion data, and a reward function based on the robot's target task and corresponding performance indicators. A virtual environment is constructed using a physics engine, and the robot's structure is designed based on the robot's target task and corresponding performance indicators. The robot's state parameters are then obtained. Control commands are generated based on the robot's state parameters and the environmental state information of the virtual environment using a Transformer network, reinforcement learning algorithm, and diagonal Gaussian policy. These control commands control the robot's movement. By combining the Transformer network, reinforcement learning algorithm, and diagonal Gaussian policy to generate control commands, efficient and precise control of the robot is achieved, accelerating algorithm iteration and robot performance optimization. This effectively solves the problems of low efficiency, poor intelligence and versatility, and difficulty in responding based on the robot's own structural information in existing technologies.

[0063] In one application scenario, Figure 4 A flowchart illustrating the robot control method is provided.

[0064] like Figure 4 As shown, the robot control method may include the following steps:

[0065] Step S1, Problem Modeling.

[0066] Specifically, the robot control problem is modeled as a reinforcement learning problem based on a diagonal Gaussian policy. This step is crucial for problem definition and modeling, as it transforms the complex robot control problem into an optimization problem within the reinforcement learning framework.

[0067] Step S2, scene virtualization.

[0068] Specifically, robot application scenarios are modeled based on a third-party physics engine, and a virtual environment similar to the real environment is created through the simulation of the physics engine, providing a foundation for subsequent robot control and testing.

[0069] Step S3, robot structure design.

[0070] Specifically, designing the robot's structure and converting the design elements into file and data formats that can be directly imported into third-party simulators is a prerequisite for robot control. It ensures that the designed robot can be accurately simulated and controlled in a virtual environment.

[0071] Step S4: Simulate scene initialization.

[0072] Specifically, the robot is imported into the simulation environment and its initial motion state is set. At the same time, the robot's structural parameters and initial motion state parameters are obtained from the simulator to provide the necessary input information for subsequent control algorithms.

[0073] Step S5, action generation.

[0074] Specifically, the Transformer control algorithm is used to receive state inputs and generate motion vectors for each joint component. This is the core of the control algorithm, which generates the optimal control commands based on the current state information.

[0075] Specifically, Transformer captures the robot's structure through position encoding or attention masking mechanisms. Position encoding refers to recording in the encoding of a joint or link which token it is connected to in another mode; attention masking refers to removing entries for unconnected components after calculating the attention association matrix.

[0076] like Figure 5 As shown, the structure of the Transformer neural network model is demonstrated, which is used to control multiple joint components of a robot. It can capture complex dependencies in the input data, thereby generating more accurate and useful outputs.

[0077] Figure 5 The multi-head attention layer receives state input (information such as the robot's current position, posture, and velocity), processes this information, and generates a series of intermediate representations. These intermediate representations are then passed to the subsequent "n layers," which contain more neural network layers (such as fully connected layers, convolutional layers, etc.) and attention layers for further processing and transformation of the data.

[0078] Ultimately, the data, after multiple layers of processing, is used to generate motion vectors for each joint component. These motion vectors are crucial for controlling the robot's joint movement, containing instructions on how the joint should move (e.g., rotation angle, movement speed). By sending these motion vectors to the robot's joint actuators, precise control of the robot's motion can be achieved.

[0079] In the above process, the state input is received through the Transformer control algorithm, and processed through a cross-head attention layer and a multi-layer neural network to finally generate the motion vector corresponding to each joint component, so as to achieve precise control of the robot's motion.

[0080] Step S6: Simulate execution and calculate rewards.

[0081] Specifically, k steps are executed in the simulator, where k equals the control frequency divided by the simulation frequency. The reward generated during the execution process is calculated using a preset reward function. This step is crucial for evaluating the performance of the control algorithm; the level of the reward reflects the quality of the control algorithm.

[0082] Step S7, motion status update.

[0083] Specifically, after the simulation is completed, the current motion state parameters of each component of the robot are retrieved from the simulator. These parameters will serve as the initial state input for the next control algorithm, forming a closed-loop control.

[0084] Step S8: Determine whether to continue the simulation.

[0085] Specifically, the system determines whether to continue the simulation based on the set conditions. If the preset number of simulations is reached or other stopping conditions are met, step S9 is executed; otherwise, the system jumps to step S5 to continue execution.

[0086] Step S9: Offline learning.

[0087] Specifically, reinforcement learning algorithms such as the Soft Actor-Critic algorithm are used to train and update the model weights. This step involves continuously learning and optimizing the control algorithm during simulation, improving its performance through iterative iteration.

[0088] The above process demonstrates the complete workflow from problem modeling to offline learning, covering all aspects of robot control problems, including scene modeling, structural design, control algorithm implementation, and performance evaluation.

[0089] Through the above process, this embodiment of the invention integrates Transformer networks and reinforcement learning algorithms to achieve efficient optimization of robot control strategies. This not only improves the accuracy and speed of robot task execution but also enhances its stability and adaptability in complex environments. At the same time, the virtual environment built using the physics engine provides a convenient platform for testing and optimizing robot control algorithms, reducing the cost and risk of actual experiments and promoting the rapid development and application of robot technology.

[0090] The following are embodiments of the apparatus of the present invention, which can be used to execute the robot control method involved in the present invention. For details not disclosed in the embodiments of the apparatus of the present invention, please refer to the method embodiments of the robot control method involved in the present invention.

[0091] Please see Figure 6 This invention provides a robot control device 800.

[0092] The device 800 includes, but is not limited to: a control target acquisition module 810, a virtual environment construction module 830, a robot structure design module 850, and a control command generation module 870.

[0093] Among them, the control target acquisition module 810 is used to acquire the robot's target task and corresponding performance indicators.

[0094] The virtual environment construction module 830 is used to obtain state data, motion data and reward function based on the robot's target task and corresponding performance indicators, and to construct a virtual environment through the physics engine.

[0095] The robot structure design module 850 is used to design the robot's structure and obtain the robot's state parameters based on the robot's target task and corresponding performance indicators.

[0096] The control command generation module 870 is used to obtain control commands based on the robot's state parameters and the environmental state information of the virtual environment through a Transformer network, reinforcement learning algorithm and diagonal Gaussian policy, and to control the robot's movement through the control commands.

[0097] It should be noted that the robot control provided in the above embodiments is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed. That is, the internal structure of the robot control device will be divided into different functional modules to complete all or part of the functions described above.

[0098] Furthermore, the robot control device and robot control method embodiments provided in the above embodiments belong to the same concept, and the specific way in which each module performs operations has been described in detail in the method embodiments, and will not be repeated here.

[0099] Figure 7 A schematic diagram of the structure of an electronic device according to an exemplary embodiment is shown.

[0100] It should be noted that this electronic device is merely an example adapted to the present invention and should not be construed as providing any limitation on the scope of use of the present invention. Furthermore, this electronic device should not be interpreted as requiring or depending on having... Figure 7 One or more components of the exemplary electronic device 2000 shown.

[0101] The hardware structure of electronic devices 2000 can vary significantly due to differences in configuration or performance, such as... Figure 7 As shown, the electronic device 2000 includes: a power supply 210, an interface 230, at least one memory 250, and at least one central processing unit (CPU) 270.

[0102] Specifically, power supply 210 is used to provide operating voltage for various hardware devices on electronic device 2000.

[0103] Interface 230 includes at least one wired or wireless network interface 231 for interacting with external devices. Of course, in other examples adapted to this invention, interface 230 may further include at least one serial-to-parallel conversion interface 233, at least one input / output interface 235, and at least one USB interface 237, etc. Figure 7 As shown, this does not constitute a specific limitation.

[0104] The memory 250 serves as a carrier for resource storage and can be a read-only memory, random access memory, disk, or optical disk, etc. The resources stored on it include the operating system 251, application programs 253, and data 255, etc., and the storage method can be temporary storage or permanent storage.

[0105] The operating system 251 is used to manage and control the various hardware devices and application programs 253 on the electronic device 2000, so as to enable the central processing unit 270 to perform calculations and processing on the massive data 255 in the memory 250. It can be Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0106] Application 253 is a computer-readable instruction based on operating system 251 that performs at least one specific task, and may include at least one module ( Figure 7 (Not shown), each module may contain computer-readable instructions for electronic device 2000. For example, a robot control device may be considered as application program 253 deployed on electronic device 2000.

[0107] Data 255 may be signal information, etc., and is stored in memory 250.

[0108] The central processing unit 270 may include one or more processors and is configured to communicate with the memory 250 via at least one communication bus to read computer-readable instructions stored in the memory 250, thereby enabling the computation and processing of massive amounts of data 255 in the memory 250. For example, a robot control method may be implemented by the central processing unit 270 reading a series of computer-readable instructions stored in the memory 250.

[0109] Furthermore, the present invention can also be implemented through hardware circuits or a combination of hardware circuits and software. Therefore, the implementation of the present invention is not limited to any specific hardware circuit, software, or combination thereof.

[0110] Please see Figure 8 This invention provides an electronic device 4000, which may include: a desktop computer, a laptop computer, a server, etc., with sensor recognition capabilities.

[0111] exist Figure 8 In this context, the electronic device 4000 includes at least one processor 4001 and at least one memory 4003.

[0112] The data interaction between the processor 4001 and the memory 4003 can be achieved through at least one communication bus 4002. This communication bus 4002 may include a path for transmitting data between the processor 4001 and the memory 4003. The communication bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The communication bus 4002 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0113] Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present invention.

[0114] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application-Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this invention. Processor 4001 may also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0115] The memory 4003 may be a ROM (Read-Only Memory) or other type of static storage device capable of storing static information and instructions, RAM (Random Access Memory) or other type of dynamic storage device capable of storing information and instructions, or it may be an EEPROM (Electrically Erasable Programmable Read-Only Memory), a CD-ROM (Compact Disc Read-Only Memory) or other optical disc storage, optical disk storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program instructions or code in the form of instructions or data structures and accessible by the electronic device 4000, but is not limited thereto.

[0116] The memory 4003 stores computer-readable instructions, and the processor 4001 can read the computer-readable instructions stored in the memory 4003 through the communication bus 4002.

[0117] The computer-readable instructions are executed by one or more processors 4001 to implement the robot control methods in the above embodiments.

[0118] Furthermore, this embodiment of the invention provides a storage medium storing computer-readable instructions, which are executed by one or more processors to implement the robot control method described above.

[0119] This invention provides a computer program product including computer-readable instructions stored in a storage medium. One or more processors of an electronic device read the computer-readable instructions from the storage medium, load and execute the computer-readable instructions, thereby enabling the electronic device to implement the robot control method described above.

[0120] Compared with related technologies, the beneficial effects of the present invention are:

[0121] 1. This invention first obtains the robot's target task and corresponding performance indicators. Based on the robot's target task and corresponding performance indicators, it obtains state data, motion data, and a reward function. A virtual environment is constructed using a physics engine. The robot's structure is designed based on the robot's target task and corresponding performance indicators. The robot's state parameters are obtained. Control commands are obtained based on the robot's state parameters and the environmental state information of the virtual environment using a Transformer network, reinforcement learning algorithm, and diagonal Gaussian policy. The robot's movement is controlled by the control commands. By combining the Transformer network, reinforcement learning algorithm, and diagonal Gaussian policy to generate control commands, efficient and precise control of the robot is achieved. This accelerates the algorithm iteration and robot performance optimization process, thereby effectively solving the problems of low efficiency, poor intelligence and versatility, and difficulty in responding based on the robot's own structural information in existing technologies.

[0122] 2. This invention significantly improves the accuracy and flexibility of robot control by introducing the Transformer control algorithm. Utilizing the powerful sequence processing and attention mechanisms of the Transformer, the model can accurately capture and analyze the robot's complex state information, thereby generating more precise and dynamic motion vectors. This not only increases the success rate of the robot in performing tasks but also enhances its adaptability and robustness in complex environments. Furthermore, this invention possesses good scalability and adjustability, facilitating optimization and improvement according to specific needs.

[0123] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0124] The above description is only a partial embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A robot control method, characterized in that, The method includes: Obtain the robot's target task and corresponding performance indicators; the target task includes path planning, object grasping, and obstacle avoidance; the performance indicators include accuracy, speed, and stability. Based on the robot's target task and corresponding performance indicators, state data, motion data, and reward function are obtained, and a virtual environment is constructed using a physics engine. The robot's structure is designed based on its target task and corresponding performance indicators, and the robot's state parameters are obtained; the structure includes links and joints. The robot's motion is controlled by obtaining control commands based on the robot's state parameters and the virtual environment's state information using a Transformer network, reinforcement learning algorithm, and diagonal Gaussian policy. The process of obtaining state data, action data, and a reward function based on the robot's target task and corresponding performance indicators includes: Based on the robot's target task and corresponding performance indicators, key variables, control commands, and reward functions are determined; the key variables are organized into state vectors as state data, and the control commands are encoded into motion vectors as motion data; the key variables include position, velocity, acceleration, and attitude; the control commands include joint rotation angles, velocities, and torques; The control commands obtained through the Transformer network, reinforcement learning algorithm, and diagonal Gaussian policy based on the robot's state parameters and the environmental state information of the virtual environment include: The state parameters and environmental state information are input into the Transformer network to obtain the mean and log-standard deviation of the diagonal Gaussian distribution; control commands are obtained based on the mean and log-standard deviation of the diagonal Gaussian distribution, and the robot is controlled to execute the control commands to obtain motion results and reward information; the parameters of the Transformer network are optimized based on the motion results and reward information using a reinforcement learning algorithm. The process of controlling the robot to execute the control commands to obtain motion results includes: The minimum distance between two components in the robot on the structure is calculated based on the motion results, and the motion results are obtained based on the minimum distance; the minimum distance is used to evaluate whether the Transformer network can capture the robot structure.

2. The robot control method as described in claim 1, characterized in that, The construction of the virtual environment through the physics engine includes: The physics engine is determined and its parameters are configured based on the robot's target task and corresponding performance indicators. A virtual environment for the robot is constructed within the physics engine; the virtual environment includes terrain, obstacles, and target objects. The robot's initial position, posture, and motion state are set in the virtual environment, and the environmental parameters and sensor data in the virtual environment are initialized.

3. The robot control method as described in claim 1, characterized in that, The step of designing the robot's structure based on the robot's target task and corresponding performance indicators includes: The structure and structural parameters of the robot are designed according to the robot's target task and corresponding performance indicators; the structural parameters include the number, size, and layout of the structure. The designed robot is converted into a file format that matches the third-party simulator and imported into the physics engine for testing. Based on the test results, the structure and structural parameters of the robot are adjusted until the test is passed.

4. The robot control method as described in claim 1, characterized in that, The method further includes: The updated Transformer network generates motion commands to control the robot's movement in the virtual environment, and the test results are obtained. Based on the test results, adjust the parameters of the Transformer network, the parameters of the reinforcement learning algorithm, the structure of the robot, and its structural parameters until the test results meet the set conditions.

5. A robot control device, characterized in that, The device is applied to the robot control method according to claim 1, and the device comprises: The target acquisition module is used to acquire the robot's target task and corresponding performance indicators; The virtual environment construction module is used to obtain state data, motion data and reward function based on the robot's target task and corresponding performance indicators, and to construct a virtual environment through a physics engine. The robot structure design module is used to design the structure of the robot according to the robot's target task and corresponding performance indicators, and to obtain the robot's state parameters. The control command generation module is used to obtain control commands based on the robot's state parameters and the environment state information of the virtual environment through a Transformer network, reinforcement learning algorithm, and diagonal Gaussian policy, and to control the robot's movement through the control commands.

6. An electronic device, characterized in that, include: At least one processor and at least one memory, wherein, The memory stores computer-readable instructions; The computer-readable instructions are executed by one or more of the processors, causing the electronic device to implement the robot control method as described in any one of claims 1 to 4.

7. A storage medium having computer-readable instructions stored thereon, characterized in that, The computer-readable instructions are executed by one or more processors to implement the robot control method as described in any one of claims 1 to 4.

Images