Robot control method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a robot control method and device, electronic equipment and a storage medium, and relates to the technical field of robots and artificial intelligence, and the method comprises the steps: obtaining a target task of a robot and a corresponding performance index, state data, action data and a reward function are obtained according to the target task of the robot and the corresponding performance index, a virtual environment is constructed through a physical engine, the structure of the robot is designed according to the target task of the robot and the corresponding performance index, and state parameters of the robot are obtained; a control instruction is obtained through a Transform network, a reinforcement learning algorithm and a diagonal Gaussian strategy according to the state parameters of the robot and the environment state information of the virtual environment, and the robot is controlled to move through the control instruction. According to the method, efficient and accurate control over the robot is achieved, and the problems that in the prior art, efficiency is low, the intelligent level and universality are poor, and response is difficult to make in combination with structural information of the robot are solved.

Description

Technical Field

[0001] The present invention relates to the field of robotics and artificial intelligence technology, and in particular to a robot control method, device, electronic equipment and storage medium. Background Art

[0002] Currently, in the field of robot control theory, although artificial intelligence (AI) technology has made remarkable progress, there are still significant defects in the universality and cutting-edge applications of robot control algorithms.

[0003] First, most of the existing robot control algorithms remain at a relatively basic technical level, such as multi-layer perceptrons (MLP) and graph neural networks. These algorithms are incapable of handling complex tasks, especially in effectively capturing the fine structural information of the robot itself and its dynamic interaction with the environment. In addition, the existing algorithms are not versatile enough and cannot be transferred between different types of robots. This limitation restricts the application effect of robots in complex and changing environments and cannot meet the growing demand for intelligence.

[0004] Secondly, although the Transformer model, as a cutting-edge achievement in the field of AI generation, has demonstrated powerful sequence modeling capabilities, its application in the field of robot control and decision-making faces many challenges. On the one hand, the Transformer model usually adopts an end-to-end token prediction method. This mechanism often leads to low execution efficiency when dealing with robot control tasks with high real-time requirements, and it is difficult to meet the needs of fast response and precise control; on the other hand, the Transformer model relies on the dynamic formula related to the robot structure when applied in the embodied intelligence algorithm, resulting in poor algorithm migration, which means that whenever facing robots with different structures or dynamic characteristics, a lot of R&D work or training processes need to be re-performed, which not only increases development costs, but also prolongs the time to market, limiting the wide application of technology. It can be seen that the existing robot control algorithms have obvious deficiencies in terms of versatility, intelligence level and execution efficiency, and it is difficult to give full play to the potential of AI technology.

[0005] Therefore, there is a need for a robot control method that can efficiently integrate AI technology and is efficient, flexible, and highly transferable. Summary of the invention

[0006] Each embodiment of the present invention provides a robot control method to solve the problems of low efficiency, poor intelligence and versatility in the prior art, and difficulty in combining the robot's own structural information to respond. The technical solution is as follows:

[0007] According to one aspect of the present invention, a robot control method comprises: obtaining a target task of the robot and a corresponding performance indicator; the target task comprises path planning, object grasping and obstacle avoidance; the performance indicators comprise accuracy, speed and stability; according to the target task of the robot and the corresponding performance indicator, state data, action data and reward function are obtained, and a virtual environment is constructed through a physical engine; according to the target task of the robot and the corresponding performance indicator, the structure of the robot is designed, and the state parameters of the robot are obtained; the structure comprises connecting rods and joints; control instructions are obtained according to the state parameters of the robot and the environmental state information of the virtual environment through a Transformer network, a reinforcement learning algorithm and a diagonal Gaussian strategy, and the movement of the robot is controlled through the control instructions.

[0008] In one of the embodiments, obtaining state data, action data and reward function according to the target task of the robot and the corresponding performance indicators is achieved by the following steps: determining key variables, control instructions and reward functions according to the target task of the robot and the corresponding performance indicators; the key variables include position, speed, acceleration, and posture; the control instructions include the rotation angle, speed and torque of the joints; organizing the key variables into state vectors as state data, and encoding the control instructions into action vectors as action data.

[0009] In one of the embodiments, constructing a virtual environment through a physical engine is achieved through the following steps: determining the physical engine and configuring parameters according to the target task of the robot and the corresponding performance indicators; constructing a virtual environment for the robot in the physical engine; the virtual environment includes terrain, obstacles and target objects; setting the initial position, posture and motion state of the robot in the virtual environment, and initializing environmental parameters and sensor data in the virtual environment.

[0010] In one of the embodiments, designing the structure of the robot according to the target task of the robot and the corresponding performance indicators is achieved by the following steps: designing the structure and structural parameters of the robot according to the target task of the robot and the corresponding performance indicators; the structural parameters include the number, size and layout of the structure; converting the designed robot into a file format that matches a third-party simulator and importing it into the physical engine for testing, and adjusting the structure and structural parameters of the robot according to the test results until the test passes.

[0011] In one of the embodiments, obtaining control instructions according to state parameters of the robot and environmental state information of the virtual environment through a Transformer network, a reinforcement learning algorithm and a diagonal Gaussian strategy is achieved by the following steps: inputting the state parameters and environmental state information into the Transformer network to obtain a mean and a logarithmic standard deviation of a diagonal Gaussian distribution; obtaining control instructions according to the mean and the logarithmic standard deviation of the diagonal Gaussian distribution, and controlling the robot to execute the control instructions to obtain motion results and reward information; and optimizing the parameters of the Transformer network according to the motion results and reward information through a reinforcement learning algorithm.

[0012] In one of the embodiments, the following steps are also included: generating action instructions through the updated Transformer network to control the robot to move in the virtual environment to obtain test results; adjusting the parameters of the Transformer network, the parameters of the reinforcement learning algorithm, the structure and structural parameters of the robot according to the test results until the test results meet the set conditions.

[0013] In one embodiment, controlling the robot to execute the control instruction to obtain a motion result is achieved by the following steps: calculating the minimum structural distance between two components in the robot according to the motion result, and obtaining the motion result according to the minimum distance; the minimum distance is used to evaluate whether the Transformer network can capture the robot structure.

[0014] According to one aspect of the present invention, a robot control device includes: a control target acquisition module, which is used to acquire the target task of the robot and the corresponding performance indicators; a virtual environment construction module, which is used to obtain state data, action data and reward functions according to the target task of the robot and the corresponding performance indicators, and to construct a virtual environment through a physical engine; a robot structure design module, which is used to design the structure of the robot according to the target task of the robot and the corresponding performance indicators, and to acquire the state parameters of the robot; a control instruction generation module, which is used to obtain control instructions according to the state parameters of the robot and the environmental state information of the virtual environment through a Transformer network, a reinforcement learning algorithm and a diagonal Gaussian strategy, and to control the movement of the robot through the control instructions.

[0015] According to one aspect of the present invention, an electronic device includes at least one processor and at least one memory, wherein the memory stores computer-readable instructions; the computer-readable instructions are executed by one or more of the processors, so that the electronic device implements the robot control method as described above.

[0016] According to one aspect of the present invention, a storage medium stores computer-readable instructions thereon, wherein the computer-readable instructions are executed by one or more processors to implement the robot control method as described above.

[0017] The beneficial effects brought by the technical solution provided by the present invention are:

[0018] In the above technical scheme, the present invention first obtains the target task and the corresponding performance indicators of the robot, obtains state data, action data and reward function according to the target task and the corresponding performance indicators of the robot, constructs a virtual environment through a physical engine, designs the structure of the robot according to the target task and the corresponding performance indicators of the robot, obtains the state parameters of the robot, obtains control instructions according to the state parameters of the robot and the environmental state information of the virtual environment through a Transformer network, a reinforcement learning algorithm and a diagonal Gaussian strategy, controls the movement of the robot through the control instructions, generates control instructions by combining the Transformer network, the reinforcement learning algorithm and the diagonal Gaussian strategy, realizes efficient and precise control of the robot, accelerates the algorithm iteration and the robot performance optimization process, thereby effectively solving the problems of low efficiency, poor intelligence level and versatility, and difficulty in responding in combination with the robot's own structural information in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in describing the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without creative work.

[0020] Figure 1 is a flow chart of a robot control method according to an exemplary embodiment;

[0021] Figure 2 yes Figure 1 A flowchart of constructing a virtual environment in step 130 in the corresponding embodiment;

[0022] Figure 3 yes Figure 1 Flow chart corresponding to step 170 in the embodiment;

[0023] Figure 4 It is a flowchart of a robot control method in an application scenario;

[0024] Figure 5 yes Figure 4 Schematic diagram of the process of generating joint motion vectors in corresponding application scenarios;

[0025] Figure 6 is a block diagram of a robot control device according to an exemplary embodiment;

[0026] Figure 7 is a hardware structure diagram of an electronic device according to an exemplary embodiment;

[0027] Figure 8 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0028] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0029] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present disclosure refers to the presence of the features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0030] The present invention provides a robot control method, which generates control instructions by combining a Transformer network, a reinforcement learning algorithm, and a diagonal Gaussian strategy, thereby achieving efficient and accurate control of the robot, solving the problems of low efficiency, poor intelligence level and versatility, and difficulty in combining the robot's own structural information to respond in the prior art. The robot control method is applicable to a robot control device, which can be an electronic device or can be run on an external host through control software, which is not limited here. The robot control method in the embodiment of the present invention can be applied to a variety of scenarios, such as robot control.

[0031] See also Figure 1 , an embodiment of the present invention provides a robot control method, which is applicable to electronic equipment.

[0032] In the following method embodiments, for ease of description, the execution subject of each step of the method is taken as an electronic device as an example for illustration, but this does not constitute a specific limitation.

[0033] like Figure 1 As shown, the method may include the following steps:

[0034] Step 110, obtaining the robot's target task and corresponding performance indicators.

[0035] Among them, the target tasks include path planning, object grasping and obstacle avoidance, etc., and the performance indicators include accuracy, speed and stability, etc., which are not limited here.

[0036] Step 130, obtain state data, action data and reward function according to the target task of the robot and the corresponding performance indicators, and build a virtual environment through a physical engine.

[0037] In one possible implementation, key variables, control instructions, and reward functions are determined according to the robot's target task and corresponding performance indicators, key variables are organized into state vectors as state data, and control instructions are encoded into action vectors as action data.

[0038] Among them, key variables include position, velocity, acceleration, posture, etc., and control instructions include joint rotation angle, velocity and torque, etc., which are not limited here.

[0039] In one possible implementation, such as Figure 2 As shown, building a virtual environment through a physical engine may include the following steps:

[0040] Step 210, determining the physical engine and configuring parameters according to the target task of the robot and the corresponding performance indicators.

[0041] Specifically, choose the most suitable physics engine to simulate the robot. The choice of physics engine depends on whether it can accurately reflect the physical interaction between the robot and the environment, such as gravity, friction, collision, etc. After selecting the physics engine, adjust the engine parameters such as time step, physical accuracy, etc. according to the specific characteristics of the robot and simulation requirements to ensure the accuracy and efficiency of the simulation.

[0042] The physical engine may include ISAC SIM, Mujoco, pymunk, etc., which is not limited here.

[0043] Step 230, constructing a virtual environment of the robot in the physical engine.

[0044] The virtual environment includes terrain, obstacles, target objects, etc., which are not limited here.

[0045] Specifically, the environment needs to simulate as closely as possible the real-life scenario in which the robot will operate. Environment construction includes adding terrain (such as flat ground, slopes, stairs, etc.), obstacles (such as walls, furniture, other robots, etc.), and target objects (such as objects that need to be grasped, locations that need to be navigated to, etc.). These elements need to be designed and placed according to actual needs to ensure that the simulation can truly reflect the various situations that the robot may encounter in the real world.

[0046] Step 250, setting the initial position, posture and motion state of the robot in the virtual environment to initialize environmental parameters and sensor data in the virtual environment.

[0047] Step 150, designing the structure of the robot according to the target task of the robot and the corresponding performance indicators, and obtaining the state parameters of the robot.

[0048] Among them, the structure of the robot includes several links and joints, which can form a tree structure, with links as nodes and joints as edges; links are ordinary rigid bodies with physical properties such as mass and volume; joints represent motion constraints between links, and have no inherent physical properties such as mass and volume, and will not collide, such as robots with URDF structure.

[0049] In one possible implementation, the structure and structural parameters of the robot are designed according to the robot's target tasks and corresponding performance indicators, the designed robot is converted into a file format that matches a third-party simulator and imported into a physical engine for testing, and the structure and structural parameters of the robot are adjusted according to the test results until the test passes.

[0050] The structural parameters include the number, size and layout of the structures, etc., which are not limited here.

[0051] Specifically, the parameters and data need to reflect the environmental conditions and perception information that the robot may encounter in actual operation to ensure the authenticity and reliability of the simulation results.

[0052] Step 170, obtaining control instructions according to the state parameters of the robot and the environmental state information of the virtual environment through the Transformer network, reinforcement learning algorithm and diagonal Gaussian strategy, and controlling the movement of the robot through the control instructions.

[0053] In one possible implementation, such as Figure 3 As shown, step 170 may include the following steps:

[0054] Step 310, input the state parameters and environmental state information into the Transformer network to obtain the mean and logarithmic standard deviation of the diagonal Gaussian distribution.

[0055] Specifically, the Transformer network processes this information through a complex self-attention mechanism, outputs the mean and logarithmic standard deviation of the diagonal Gaussian distribution of the control strategy, and provides a probability distribution basis for the generation of control instructions.

[0056] Step 330, obtaining a control instruction according to the mean and logarithmic standard deviation of the diagonal Gaussian distribution, and controlling the robot to execute the control instruction to obtain a motion result and reward information.

[0057] Specifically, according to the mean and logarithmic standard deviation of the diagonal Gaussian distribution output by the Transformer network, sampling is performed to obtain specific control instructions. These control instructions include parameters such as the robot's movement direction and speed, which are used to guide the robot's actual movement in the virtual environment. After executing the control instructions, the robot's movement results (such as the final position, posture changes, etc.) and the resulting reward information (such as whether the task is successfully completed, whether a collision occurs, etc.) are observed and recorded.

[0058] Step 350, optimizing the parameters of the Transformer network according to the movement results and reward information through a reinforcement learning algorithm.

[0059] Specifically, reinforcement learning algorithms (such as Soft Actor-Cr itic, etc.) are used to optimize the parameters of the Transformer network based on the robot's motion results and reward information in the virtual environment. By continuously adjusting the network parameters, the robot can obtain higher cumulative rewards when performing the same task, thereby improving the robot's autonomous decision-making and motion control capabilities in complex environments. This process is an iterative cycle, and through continuous trial and error and learning, it gradually approaches the optimal control strategy.

[0060] In one possible implementation, the updated Transformer network generates action instructions to control the robot to move in a virtual environment to obtain test results, and the parameters of the Transformer network, the parameters of the reinforcement learning algorithm, and the structure and structural parameters of the robot are adjusted according to the test results until the test results meet the set conditions.

[0061] In one possible implementation, the minimum structural distance between two components in the robot is calculated based on the motion result, and the motion result is obtained based on the minimum distance, wherein the minimum distance is used to evaluate whether the Transformer network can capture the robot structure.

[0062] Through the above process, the present invention first obtains the target task and the corresponding performance indicators of the robot, obtains state data, action data and reward function according to the target task and the corresponding performance indicators of the robot, constructs a virtual environment through a physical engine, designs the structure of the robot according to the target task and the corresponding performance indicators of the robot, obtains the state parameters of the robot, obtains control instructions according to the state parameters of the robot and the environmental state information of the virtual environment through a Transformer network, a reinforcement learning algorithm and a diagonal Gaussian strategy, controls the movement of the robot through the control instructions, generates control instructions by combining the Transformer network, the reinforcement learning algorithm and the diagonal Gaussian strategy, realizes efficient and accurate control of the robot, accelerates the algorithm iteration and the robot performance optimization process, thereby effectively solving the problems of low efficiency, poor intelligence level and versatility, and difficulty in responding in combination with the robot's own structural information in the prior art.

[0063] In one application scenario, Figure 4 A flow chart showing the robot control method.

[0064] like Figure 4 As shown, the robot control method may include the following steps:

[0065] Step S1, problem modeling.

[0066] Specifically, the robot control problem is modeled as a reinforcement learning problem based on a diagonal Gaussian strategy. This step is the key to problem definition and modeling, which transforms the complex robot control problem into an optimization problem under the reinforcement learning framework.

[0067] Step S2: scene virtualization.

[0068] Specifically, the robot application scenario is modeled based on a third-party physics engine, and a virtual environment similar to the real environment is created through the simulation of the physics engine, providing a basis for subsequent robot control and testing.

[0069] Step S3, robot structure design.

[0070] Specifically, designing the robot's structure and converting the design elements into file formats and data formats that can be directly imported into third-party simulators is a prerequisite for robot control, which ensures that the designed robot can be accurately simulated and controlled in a virtual environment.

[0071] Step S4, initializing the simulation scene.

[0072] Specifically, the robot is imported into the simulation environment and its initial motion state is set. At the same time, the structural parameters and initial motion state parameters of the robot are obtained from the simulator to provide necessary input information for the subsequent control algorithm.

[0073] Step S5, action generation.

[0074] Specifically, the Transformer control algorithm is used to receive state input to generate the action vector corresponding to each joint component. This is the core part of the control algorithm, which generates the optimal control instructions based on the current state information.

[0075] Specifically, Transformer captures the robot structure through position encoding or attention mask mechanism. Position encoding means that in the encoding of a joint or link, it is recorded that it is connected to the component of the token of another modality; attention mask means that after calculating the attention association matrix, the entries of unconnected components are deleted.

[0076] like Figure 5 As shown in the figure, the structure of the Transformer neural network model is shown, which is used to control multiple joint components of the robot. It can capture the complex dependencies in the input data to generate more accurate and useful outputs.

[0077] Figure 5 The cross-headed attention layer receives state input (the robot’s current position, posture, speed, etc.), then processes this information and generates a series of intermediate representations, which are then passed to the subsequent "n layers", which contain more neural network layers (such as fully connected layers, convolutional layers, etc.) and attention layers for further processing and transformation of data.

[0078] Finally, the data after multi-layer processing is used to generate the action vector corresponding to each joint component. The action vector is the key information for controlling the movement of the robot joints, and contains instructions on how the joints should move (such as rotation angle, movement speed, etc.). By sending these action vectors to the robot's joint drivers, precise control of the robot's movement can be achieved.

[0079] In the above process, the state input is received through the Transformer control algorithm, and the action vector corresponding to each joint component is finally generated through cross-multi-head attention layer and multi-layer neural network processing to achieve precise control of the robot movement.

[0080] Step S6, simulation execution and reward calculation.

[0081] Specifically, k steps are executed in the simulator, where k is equal to the control frequency divided by the simulation frequency, and the reward generated by the execution process is calculated through the preset reward function. This step is the key to evaluating the performance of the control algorithm, and the high or low reward value reflects the quality of the control algorithm.

[0082] Step S7, motion status update.

[0083] Specifically, after the simulation is executed, the current motion state parameters of each component of the robot are retrieved from the simulator. These parameters will be used as the initial state of the next control algorithm input to form a closed-loop control.

[0084] Step S8, determining whether to continue the simulation.

[0085] Specifically, it is determined whether the simulation needs to be continued according to the set conditions. If the preset number of simulations is reached or other stop conditions are met, step S9 is executed; otherwise, the process jumps to step S5 and continues to execute.

[0086] Step S9: offline learning.

[0087] Specifically, a reinforcement learning algorithm such as the Soft Actor-Critic algorithm is used to train and update the model weights. This step is the process of continuously learning and optimizing the control algorithm during the simulation, and improving the performance of the control algorithm through repeated iterations.

[0088] The above process demonstrates the complete flow from problem modeling to offline learning, covering all aspects of robot control problems, including scenario modeling, structure design, control algorithm implementation, and performance evaluation.

[0089] Through the above process, the embodiment of the present invention realizes efficient optimization of the robot control strategy by integrating the Transformer network and the reinforcement learning algorithm, which not only improves the accuracy and speed of the robot in executing tasks, but also enhances its stability and adaptability in complex environments. At the same time, the virtual environment constructed by the physical engine provides a convenient platform for testing and optimization of the robot control algorithm, reduces the cost and risk of actual experiments, and promotes the rapid development and application promotion of robot technology.

[0090] The following is an embodiment of the device of the present invention, which can be used to execute the robot control method involved in the present invention. For details not disclosed in the embodiment of the device of the present invention, please refer to the method embodiment of the robot control method involved in the present invention.

[0091] See also Figure 6 , a robot control device 800 is provided in an embodiment of the present invention.

[0092] The device 800 includes but is not limited to: a control target acquisition module 810 , a virtual environment construction module 830 , a robot structure design module 850 and a control instruction generation module 870 .

[0093] The control target acquisition module 810 is used to acquire the target task of the robot and the corresponding performance index.

[0094] The virtual environment construction module 830 is used to obtain state data, action data and reward function according to the target task of the robot and the corresponding performance indicators, and to construct a virtual environment through a physical engine.

[0095] The robot structure design module 850 is used to design the structure of the robot according to the target tasks of the robot and the corresponding performance indicators, and obtain the state parameters of the robot.

[0096] The control instruction generation module 870 is used to obtain control instructions according to the state parameters of the robot and the environmental state information of the virtual environment through the Transformer network, reinforcement learning algorithm and diagonal Gaussian strategy, and control the movement of the robot through the control instructions.

[0097] It should be noted that the robot control provided in the above embodiments is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the robot control device will be divided into different functional modules to complete all or part of the functions described above.

[0098] In addition, the robot control device and the robot control method provided in the above embodiments belong to the same concept, and the specific manner in which each module performs operations has been described in detail in the method embodiments and will not be repeated here.

[0099] Figure 7 The present invention schematically illustrates the structure of an electronic device according to an exemplary embodiment.

[0100] It should be noted that the electronic device is only an example adapted to the present invention and cannot be considered to provide any limitation on the scope of use of the present invention. The electronic device cannot be interpreted as requiring dependence on or having Figure 7 One or more components of exemplary electronic device 2000 are shown.

[0101] The hardware structure of the electronic device 2000 may vary greatly due to differences in configuration or performance, such as Figure 7 As shown, the electronic device 2000 includes: a power supply 210, an interface 230, at least one memory 250, and at least one central processing unit (CPU) 270.

[0102] Specifically, the power supply 210 is used to provide operating voltage for each hardware device on the electronic device 2000 .

[0103] The interface 230 includes at least one wired or wireless network interface 231 for interacting with external devices. Of course, in other examples adapted by the present invention, the interface 230 may further include at least one serial-to-parallel conversion interface 233, at least one input-output interface 235, and at least one USB interface 237, etc. Figure 7 As shown, this is not intended to be a specific limitation.

[0104] The memory 250 is a carrier for storing resources, which may be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon include an operating system 251, an application 253 and data 255, etc. The storage method may be temporary storage or permanent storage.

[0105] Among them, the operating system 251 is used to manage and control various hardware devices and application programs 253 on the electronic device 2000 to enable the central processing unit 270 to calculate and process the massive data 255 in the memory 250. It can be Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0106] The application program 253 is a computer-readable instruction that performs at least one specific task based on the operating system 251, and may include at least one module ( Figure 7 (not shown), each module may respectively contain computer-readable instructions for the electronic device 2000. For example, the robot control device may be regarded as an application 253 deployed on the electronic device 2000.

[0107] The data 255 may be signal information, etc., stored in the memory 250 .

[0108] The central processor 270 may include one or more processors and is configured to communicate with the memory 250 through at least one communication bus to read the computer-readable instructions stored in the memory 250, thereby realizing the operation and processing of the mass data 255 in the memory 250. For example, the robot control method is completed by the central processor 270 reading a series of computer-readable instructions stored in the memory 250.

[0109] In addition, the present invention can also be implemented by hardware circuits or hardware circuits combined with software. Therefore, the implementation of the present invention is not limited to any specific hardware circuits, software, or a combination of the two.

[0110] See also Figure 8 In an embodiment of the present invention, an electronic device 4000 is provided. The electronic device 4000 may include: a desktop computer, a laptop computer, a server, etc. with sensor recognition capability.

[0111] exist Figure 8 In the embodiment, the electronic device 4000 includes at least one processor 4001 and at least one memory 4003.

[0112] The data interaction between the processor 4001 and the memory 4003 can be realized through at least one communication bus 4002. The communication bus 4002 may include a path for transmitting data between the processor 4001 and the memory 4003. The communication bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The communication bus 4002 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0113] Optionally, the electronic device 4000 may further include a transceiver 4004, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception, etc. It should be noted that in actual applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present invention.

[0114] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of the present invention. Processor 4001 may also be a combination that implements a computing function, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0115] The memory 4003 can be a ROM (Read On Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrical Erasable Programmable Read On Memory), a CD-ROM (Compact Disk Read On Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program instructions or codes in the form of instructions or data structures and can be accessed by the electronic device 4000, but is not limited to this.

[0116] Computer-readable instructions are stored in the memory 4003 , and the processor 4001 can read the computer-readable instructions stored in the memory 4003 through the communication bus 4002 .

[0117] The computer-readable instructions are executed by one or more processors 4001 to implement the robot control method in the above-mentioned embodiments.

[0118] In addition, an embodiment of the present invention provides a storage medium on which computer-readable instructions are stored. The computer-readable instructions are executed by one or more processors to implement the robot control method as described above.

[0119] A computer program product is provided in an embodiment of the present invention. The computer program product includes computer-readable instructions, and the computer-readable instructions are stored in a storage medium. One or more processors of an electronic device read the computer-readable instructions from the storage medium, load and execute the computer-readable instructions, so that the electronic device implements the robot control method as described above.

[0120] Compared with the related art, the present invention has the following beneficial effects:

[0121] 1. The present invention first obtains the target task and corresponding performance indicators of the robot, obtains state data, action data and reward function according to the target task and corresponding performance indicators of the robot, constructs a virtual environment through a physical engine, designs the structure of the robot according to the target task and corresponding performance indicators of the robot, obtains the state parameters of the robot, obtains control instructions according to the state parameters of the robot and the environmental state information of the virtual environment through a Transformer network, a reinforcement learning algorithm and a diagonal Gaussian strategy, controls the movement of the robot through the control instructions, generates control instructions by combining the Transformer network, the reinforcement learning algorithm and the diagonal Gaussian strategy, realizes efficient and accurate control of the robot, accelerates the algorithm iteration and the robot performance optimization process, thereby effectively solving the problems of low efficiency, poor intelligence level and versatility, and difficulty in responding in combination with the robot's own structural information in the prior art.

[0122] 2. The present invention significantly improves the accuracy and flexibility of robot control by introducing the Transformer control algorithm. By using the powerful sequence processing and attention mechanism of Transformer, the model can accurately capture and analyze the complex state information of the robot, thereby generating more accurate and dynamic action vectors. This not only improves the success rate of the robot in performing tasks, but also enhances its adaptability and robustness in complex environments. In addition, the present invention also has good scalability and adjustability, which is convenient for optimization and improvement according to specific needs.

[0123] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0124] The above descriptions are only some embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A robot control method, characterized in that: The method comprises: Obtaining the robot's target tasks and corresponding performance indicators; the target tasks include path planning, object grasping and obstacle avoidance; the performance indicators include accuracy, speed and stability; According to the target task of the robot and the corresponding performance indicators, state data, action data and reward function are obtained, and a virtual environment is constructed through a physical engine; Design the structure of the robot according to the target task of the robot and the corresponding performance indicators, and obtain the state parameters of the robot; the structure includes connecting rods and joints; A control instruction is obtained according to the state parameters of the robot and the environmental state information of the virtual environment through a Transformer network, a reinforcement learning algorithm and a diagonal Gaussian strategy, and the movement of the robot is controlled by the control instruction.

2. The robot control method according to claim 1, characterized in that: The state data, action data and reward function are obtained according to the target task of the robot and the corresponding performance index, including: Determine key variables, control instructions and reward functions according to the target tasks of the robot and the corresponding performance indicators; the key variables include position, speed, acceleration, and posture; the control instructions include the rotation angle, speed and torque of the joints; The key variables are organized into state vectors as state data, and the control instructions are encoded into action vectors as action data.

3. The robot control method according to claim 1, characterized in that: The virtual environment is constructed by using a physical engine, including: Determine the physical engine and configure parameters according to the target task of the robot and the corresponding performance indicators; Constructing a virtual environment of the robot in the physical engine; the virtual environment includes terrain, obstacles and target objects; The initial position, posture and motion state of the robot are set in the virtual environment, and the environmental parameters and sensor data in the virtual environment are initialized.

4. The robot control method according to claim 1, characterized in that: The designing of the structure of the robot according to the target task of the robot and the corresponding performance index comprises: Design the structure and structural parameters of the robot according to the target tasks of the robot and the corresponding performance indicators; the structural parameters include the number, size and layout of the structure; The designed robot is converted into a file format that matches a third-party simulator and imported into the physical engine for testing. The structure and structural parameters of the robot are adjusted according to the test results until the test passes.

5. The robot control method according to claim 1, characterized in that: The method of obtaining control instructions according to the state parameters of the robot and the environmental state information of the virtual environment through the Transformer network, the reinforcement learning algorithm and the diagonal Gaussian strategy includes: Inputting the state parameters and the environment state information into the Transformer network to obtain the mean and logarithmic standard deviation of the diagonal Gaussian distribution; Obtaining a control instruction according to the mean and logarithmic standard deviation of the diagonal Gaussian distribution, and controlling the robot to execute the control instruction to obtain a motion result and reward information; The parameters of the Transformer network are optimized according to the movement results and reward information through a reinforcement learning algorithm.

6. The robot control method according to claim 5, characterized in that: The method further comprises: Generate action instructions through the updated Transformer network to control the robot to move in the virtual environment to obtain test results; According to the test results, the parameters of the Transformer network, the parameters of the reinforcement learning algorithm, the structure and structural parameters of the robot are adjusted until the test results meet the set conditions.

7. The robot control method according to claim 5, characterized in that: The controlling the robot to execute the control instruction to obtain a motion result includes: The minimum structural distance between two components in the robot is calculated according to the motion result, and the motion result is obtained according to the minimum distance; the minimum distance is used to evaluate whether the Transformer network can capture the robot structure.

8. A robot control device, characterized in that: The device comprises: The control target acquisition module is used to obtain the robot's target task and corresponding performance indicators; A virtual environment construction module, used to obtain state data, action data and reward function according to the target task of the robot and the corresponding performance indicators, and to construct a virtual environment through a physical engine; A robot structure design module, used to design the structure of the robot according to the target task of the robot and the corresponding performance indicators, and obtain the state parameters of the robot; A control instruction generation module is used to obtain control instructions according to the state parameters of the robot and the environmental state information of the virtual environment through a Transformer network, a reinforcement learning algorithm and a diagonal Gaussian strategy, and control the movement of the robot through the control instructions.

9. An electronic device, characterized in that: include: at least one processor and at least one memory, wherein: The memory has computer-readable instructions stored thereon; The computer-readable instructions are executed by one or more of the processors so that the electronic device implements the robot control method according to any one of claims 1 to 7.

10. A storage medium having computer-readable instructions stored thereon, characterized in that: The computer-readable instructions are executed by one or more processors to implement the robot control method according to any one of claims 1 to 7.

Images