Robot control and training methods and devices based on adaptive swarm intelligence

Abstract

This invention discloses an adaptive control method, training method, and apparatus for swarm intelligence agents, relating to the fields of artificial intelligence and computer science. The method includes: constructing joint intelligence agents for each joint of a robot and configuring connection relationships; constructing a central intelligence agent to process input information; setting a mechanism for parameter sharing and synchronous updating for each joint intelligence agent and the central intelligence agent; the central intelligence agent outputting signals to control each joint intelligence agent; acquiring real motion data and evaluation data; and training and optimizing each joint intelligence agent and the central intelligence agent based on the real motion data and evaluation data through imitation learning and reinforcement learning before controlling the robot. This invention solves the problems of existing technologies' inability to achieve multi-task, multi-scenario robot control, and poor generalization and flexibility, by constructing intelligence agents for each joint of the robot and configuring connection relationships, and by constructing a central intelligence agent to process diverse input information.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and computer science, and in particular to an adaptive control method, training method and apparatus for swarm intelligence agents. Background Technology

[0002] With the development of multimodal intelligent agents, current robots already possess general language and image understanding capabilities. However, many challenges remain in robot control. These challenges mainly include the problems of robot sensors and structural heterogeneity. Different robots require different control strategies and have different dimensions of control variables, making it difficult to handle highly diverse and complex mechanical control within a single intelligent agent.

[0003] Furthermore, it is difficult to understand that the signals at the cognitive and execution levels are quite different and cannot be considered within the same agent. When using hierarchical agents for control, it is difficult to define the task division and communication methods of different agent layers. Artificial definition can easily lead to loss of effectiveness and error propagation.

[0004] Meanwhile, current robot control methods are difficult to use for multiple tasks and multiple scenarios. For each task and scenario, a large amount of data must be collected in advance, and they do not have the ability to generalize out-of-distribution.

[0005] Therefore, there is a need for a robot control method that can achieve complex, nonlinear, multi-joint robot control, has language, text, and image interaction capabilities, as well as multi-scene and multi-task generalization capabilities, and can adapt to heterogeneous robots and different control requirements. Summary of the Invention

[0006] The embodiments of this invention provide a robot control method based on adaptive swarm intelligence to solve the problems of existing technologies that cannot achieve multi-task, multi-scenario robot control, and have poor generalization ability and flexibility. The technical solution is as follows:

[0007] According to one aspect of the present invention, a robot control method based on adaptive swarm intelligence agents is provided. The method includes: constructing joint intelligence agents for each joint of the robot and configuring the connection relationships between the joint intelligence agents; the joint intelligence agents include arm intelligence agents and waist intelligence agents; constructing a central intelligence agent for the robot, and processing input information through the central intelligence agent; the input information includes image signals, text signals, and voice signals; setting a mechanism for parameter sharing and synchronous updating for each joint intelligence agent and the central intelligence agent; the central intelligence agent is used to output signals to control each joint intelligence agent; acquiring real motion data and evaluation data, and training and optimizing each joint intelligence agent and the central intelligence agent through imitation learning and reinforcement learning based on the real motion data and evaluation data before performing robot control.

[0008] In one embodiment, the arm agent is connected to the finger root agent and the finger joint agent, and the waist agent is connected to the hip joint agent, the knee joint agent and the ankle joint agent.

[0009] In one embodiment, the formula for information transfer between the interconnected intelligent agents is as follows:

[0010] ;

[0011] in, Indicates the first Parameters of an agent express The memory tensor of a moment, express The output of the intelligent agent at any time, Indicates the first The structure of an intelligent agent express Input at any moment.

[0012] In one embodiment, a central agent is constructed for the robot, and the central agent processes input information through the following steps: if the input information is a text signal, the text signal is processed to obtain a symbol sequence, and the symbol sequence is converted into a vector sequence through the embedding coding layer of the central agent; if the input information is an image signal, the image signal is encoded into a vector sequence through a convolutional neural network; if the input information is a speech signal, the speech signal is processed into a vector sequence through a speech processing algorithm and a sequence coding network.

[0013] In one embodiment, acquiring real motion data and evaluation data, and training and optimizing the joint agents and central agent based on the real motion data and evaluation data through imitation learning and reinforcement learning, is achieved through the following steps: acquiring real motion data and evaluation data of each robot type performing tasks; the robot type includes wheeled, wheeled-legged, quadrupedal, and bipedal; the tasks include gripper grasping, dexterous hand grasping, and navigation object finding; training the joint agents and central agent based on the real motion data using an imitation learning algorithm; and training the joint agents and central agent based on the real motion data and evaluation data using a reinforcement learning algorithm.

[0014] According to one aspect of the present invention, a robot training method based on adaptive swarming agents is provided. The method includes: collecting training data; the training data includes expert demonstration data and automated strategy data; the expert demonstration data is data generated by experts demonstrating the execution of tasks; the automated strategy data is data generated by agents performing tasks; during the training process, an expert evaluation mechanism is introduced to automatically evaluate the agents, and the parameters of each agent are optimized by reducing the error between the expert demonstration actions and the output of the agents, and the agents are trained through imitation learning; during the training process, an automated evaluation mechanism is introduced to score the input and output values ​​recorded in each agent, and reinforcement learning is used to optimize each agent based on the scoring results, and the agents are trained through reinforcement learning.

[0015] In one embodiment, the parameters of each agent are optimized by reducing the error between the expert demonstration action and the agent output through the following steps: inputting the input information recorded in the expert demonstration data into the agent for operation, obtaining the output prediction based on the operation data of each agent at each time point, and calculating the loss value for training based on the output prediction and the output information recorded in the expert demonstration.

[0016] The specific formula for calculating the loss value is as follows:

[0017]

[0018] in, This represents the loss value of imitation learning. This represents the input index of the expert demonstration data. Indicates time, This represents the agent's output prediction. This represents the output data of the expert demonstration.

[0019] In one embodiment, the optimization of each agent using reinforcement learning based on the scoring results is achieved through the following steps: inputting the inputs and outputs recorded in each agent into an automated evaluation system to obtain evaluation values; inputting the input values ​​recorded in each agent into the agent to obtain output predictions; and calculating loss values ​​based on the output predictions, the recorded outputs, and the evaluation values ​​for training.

[0020] The specific formula for calculating the loss value is as follows:

[0021]

[0022] in, This represents the loss value for reinforcement learning. It is the attenuation coefficient, which takes a value between 0.95 and 0.99.

[0023] In one embodiment, the method further includes: acquiring a central training set, initializing the node neural network in the central agent, and setting the shared attributes of the node neural network parameters; the central training set includes a large amount of corpus, images, videos, and multimodal alignment data; when optimizing the loss value, the gradient is propagated through backpropagation, and gradient transmission is carried out using communication between nodes so that all agent parameters are optimized; the central agent is warm-started using a multimodal pre-trained model, the central agent is pre-trained using the central training set, and the parameters of the central agent are optimized according to the input and output of the central agent at each time step.

[0024] According to one aspect of the present invention, a robot control device includes: a joint agent construction module for constructing joint agents for each joint of the robot and configuring the connection relationships between the joint agents; the joint agents include an arm agent and a waist agent; a central agent construction module for constructing a central agent for the robot and processing input information through the central agent; the input information includes image signals, text signals, and voice signals; a parameter mechanism setting module for setting a parameter sharing and synchronous update mechanism for each joint agent and the central agent; the central agent is used to output signals to control each joint agent; and a robot training module for acquiring real motion data and evaluation data, and training and optimizing each joint agent and the central agent through imitation learning and reinforcement learning based on the real motion data and evaluation data before controlling the robot.

[0025] The beneficial effects of the technical solution provided by this invention are:

[0026] In the above technical solution, the present invention first constructs joint agents for each joint of the robot and configures the connection relationships between the joint agents, constructs a central agent for the robot, and processes input information through the central agent, including image signals, text signals, and voice signals. A mechanism for parameter sharing and synchronous updating is set for each joint agent and the central agent. The central agent is used to output signals to control each joint agent, acquire real motion data and evaluation data, and trains and optimizes each joint agent and the central agent based on the real motion data and evaluation data through imitation learning and reinforcement learning before controlling the robot. This is achieved through an adaptive swarm intelligence robot control method. This method significantly enhances the flexibility and intelligence of robot behavior. By constructing intelligent agents for each joint of the robot and configuring their connections, as well as building a central intelligent agent to process diverse input information, it strengthens the robot's environmental adaptability and decision-making efficiency. The parameter sharing and synchronous update mechanism ensures collaborative work among intelligent agents. Combined with imitation learning and reinforcement learning, the robot can perform self-optimization based on real motion data and evaluation data, improving control accuracy and response speed. It provides an efficient and intelligent solution for robot applications in complex environments, effectively addressing the problems of existing technologies being unable to achieve multi-task, multi-scenario robot control, and having poor generalization ability and flexibility. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart illustrating a robot control method based on an adaptive swarming agent according to an exemplary embodiment;

[0029] Figure 2 This is an exemplary embodiment illustrating the topological structure of inputs and outputs between various intelligent agents;

[0030] Figure 3 This is a block diagram illustrating a robot control device according to an exemplary embodiment. Detailed Implementation

[0031] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0032] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this disclosure means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0033] To address this, the present invention provides a robot control method based on adaptive swarm intelligence. By constructing sophisticated joint intelligences and a central intelligence and configuring their connection relationships, precise and flexible robot motion control is achieved. This effectively solves the problems of existing technologies, such as the inability to achieve multi-task and multi-scenario robot control, and poor generalization and flexibility. This robot control method based on adaptive swarm intelligence is applicable to robot control devices, which can be electronic devices. The robot control method based on adaptive swarm intelligence in the embodiments of the present invention can be applied to various scenarios, such as robot control.

[0034] Please see Figure 1 This invention provides a robot control method based on adaptive swarm intelligence, which is applicable to electronic devices.

[0035] In the following method embodiments, for ease of description, the execution subject of each step of the method is an electronic device, but this does not constitute a specific limitation.

[0036] like Figure 1 As shown, the method may include the following steps:

[0037] Step 110: Build joint agents for each joint of the robot and configure the connection relationships between the joint agents.

[0038] Among them, the joint intelligent agent includes the arm intelligent agent and the waist intelligent agent.

[0039] In one possible implementation, the arm agent connects to the finger root agent and the finger joint agent, and the waist agent connects to the hip joint agent, the knee joint agent, and the ankle joint agent.

[0040] In one possible implementation, the formula for information transfer between interconnected agents is as follows:

[0041] ;

[0042] in, Indicates the first Parameters of an agent express The memory tensor of a moment, express The output of the intelligent agent at any time, Indicates the first The structure of an intelligent agent express Input at any moment.

[0043] Step 130: Construct a central intelligent agent for the robot, and process the input information through the central intelligent agent.

[0044] The input information includes image signals, text signals, and voice signals.

[0045] In one possible implementation, if the input information is a text signal, the text signal is processed to obtain a symbol sequence, and the symbol sequence is converted into a vector sequence through the embedding coding layer of the central agent; if the input information is an image signal, the image signal is encoded into a vector sequence through a convolutional neural network; if the input information is a speech signal, the speech signal is processed into a vector sequence through a speech processing algorithm and a sequence coding network.

[0046] Step 150: Set up a mechanism for parameter sharing and synchronous updates for each joint agent and the central agent.

[0047] Among them, the central intelligent agent is used to output signals to control the intelligent agents of each joint.

[0048] Step 170: Obtain real motion data and evaluation data. Based on the real motion data and evaluation data, train and optimize the joint agents and central agent through imitation learning and reinforcement learning, and then control the robot.

[0049] In one possible implementation, real motion data and evaluation data of each robot model performing tasks are obtained. Imitation learning algorithms are used to train the joint agents and central agents based on the real motion data, and reinforcement learning algorithms are used to train the joint agents and central agents based on the real motion data and evaluation data.

[0050] The models include wheeled, wheeled, quadrupedal, and bipedal types, and the tasks include gripper grasping, dexterous hand grasping, and navigation and object retrieval, etc., without any specific limitations.

[0051] In one possible implementation, this embodiment of the invention provides a robot training method applied to the above-mentioned adaptive swarm intelligence agent. Training data is collected, and an expert evaluation mechanism is introduced during training to automatically evaluate the intelligence agents. The parameters of each intelligence agent are optimized by reducing the error between the expert demonstration actions and the agent's output, and each intelligence agent undergoes imitation learning training. During training, an automated evaluation mechanism is introduced to score the input and output values ​​recorded in each intelligence agent. Based on the scoring results, reinforcement learning is used to optimize each intelligence agent, and each intelligence agent undergoes reinforcement learning training.

[0052] The training data includes expert demonstration data and automated policy data. Expert demonstration data is the data generated by experts demonstrating the execution of tasks, while automated policy data is the data generated by the agent's execution.

[0053] In one possible implementation, the input information recorded in the expert demonstration data is input into the agent for operation. The output prediction is obtained based on the operation data of each agent at each time step. The loss value is calculated based on the output prediction and the output information recorded in the expert demonstration for training.

[0054] The specific formula for calculating the loss value is as follows:

[0055]

[0056] in, This represents the loss value of imitation learning. This represents the input index of the expert demonstration data. Indicates time, This represents the agent's output prediction. This represents the output data of the expert demonstration.

[0057] In one possible implementation, the inputs and outputs recorded by each agent are fed into an automated evaluation system to obtain evaluation values. The input values ​​recorded by each agent are fed into the agent to obtain output predictions. Based on the output predictions, the recorded outputs, and the evaluation values, a loss value is calculated for training.

[0058] The specific formula for calculating the loss value is as follows:

[0059]

[0060] in, This represents the loss value for reinforcement learning. It is the attenuation coefficient, which takes a value between 0.95 and 0.99.

[0061] In one possible implementation, a central training set is obtained, the node neural networks in the central agent are initialized, the shared attributes of the node neural network parameters are set, and when the loss value is optimized, the gradient is propagated through backpropagation. Gradient transfer is carried out by communication between nodes so that all agent parameters are optimized. A multimodal pre-trained model is used to perform a warm start on the central agent, and the central agent is pre-trained using the central training set. The parameters of the central agent are optimized based on the input and output of the central agent at each time step.

[0062] The central training set includes a large amount of corpus, images, videos, and multimodal alignment data.

[0063] Through the above process, this embodiment of the invention first constructs joint agents for each joint of the robot and configures the connection relationships between them, then constructs a central agent for the robot. The central agent processes input information, including image signals, text signals, and voice signals. A mechanism for parameter sharing and synchronous updating is set up for each joint agent and the central agent. The central agent outputs signals to control each joint agent, acquiring real motion data and evaluation data. Based on the real motion data and evaluation data, the joint agents and the central agent are trained and optimized through imitation learning and reinforcement learning before robot control is achieved. This is achieved through adaptive swarm intelligence-based robot control. This method significantly enhances the flexibility and intelligence of robot behavior. By constructing intelligent agents for each joint of the robot and configuring their connections, as well as building a central intelligent agent to process diverse input information, it improves the robot's environmental adaptability and decision-making efficiency. The parameter sharing and synchronous update mechanism ensures collaborative work among the intelligent agents. Combined with imitation learning and reinforcement learning, the robot can perform self-optimization based on real motion data and evaluation data, improving control accuracy and response speed. This provides an efficient and intelligent solution for robot applications in complex environments, effectively addressing the problems of existing technologies being unable to achieve multi-task and multi-scenario robot control, and having poor generalization ability and flexibility.

[0064] In one exemplary embodiment, the present invention provides a robot control method based on adaptive swarm intelligence for robot control.

[0065] The central intelligence agent is bound to the robot's brain or eyes. A robotic arm intelligence agent and a robotic leg root intelligence agent are created for the robot's hands and legs, respectively. To support hand movements, a finger root intelligence agent and a finger joint intelligence agent are set for each finger (assuming each finger has 2 joints). To support leg movements, a hip joint intelligence agent, a knee joint intelligence agent, and an ankle joint intelligence agent are set in sequence.

[0066] Furthermore, in this modular design, gradient information is passed between the agents, as shown in the following formula:

[0067] ;

[0068] in, Indicates the first Parameters of an agent express The memory tensor of a moment, express The output of the intelligent agent at any time, Indicates the first The structure of an intelligent agent express Input at any moment.

[0069] Except for the central intelligence body, all other joint control intelligence bodies use the same intelligence body structure. and the same parameters .

[0070] It is worth noting that the aforementioned agents can include structures such as RNN, LSTM, Neural Turing Machine, Transformer, and Mamba. The specific agent structure is not limited here.

[0071] Taking RNN / LSTM / Mamba and other structures as examples, This represents a sequence of vectors, where each vector represents a symbol. It is the output, shape and They should be completely consistent; The corresponding hidden state is updated every time a piece of input is received.

[0072] Taking Transformer as an example, This corresponds to the Key-Value Cache, which expands upon each inference.

[0073] Specifically, the memory tensor is initialized during each deployment. During inference, the memory tensor of each agent is updated, which allows the agent to adapt to different joints.

[0074] Specifically, the input signals mainly include raw sensor signals or sensor signals after preliminary processing, and other related agent outputs. These include, but are not limited to: text signals, which are initially processed into a series of symbols and then converted into vector sequences through an embedding layer; image signals, which are encoded into vector matrices and sequences using convolutional neural networks, etc.; and speech signals, which are processed into vector sequences using speech processing methods or sequence coding networks. The vector sequences of related agent outputs are prefixed with the corresponding agent identifier. If multiple related agent outputs are received, these vector sequences are concatenated end-to-end to form a new vector sequence as the output.

[0075] Specifically, the output signals include communication information from other associated agents and terminal output signals. If the output contains a specific prefix (an identifier prefix for the associated agent), the following information is considered a signal output to the associated agent, until a terminator is encountered, indicating the end of the signal output to that agent. Terminal output signals include voice, text, and actuator control signals, etc. The central agent is responsible for voice and text output, while other node networks output actuator control signals to control the robot's movement.

[0076] like Figure 2 The diagram illustrates the input-output topology between the agents. The topology diagram presents a complex network connection, including original input 1, node 1, node 2, and node 3, and the connections between them. Node 1 transmits information to original input 1 and node 2, node 2 transmits information to node 3, and terminal 4 and terminal 5 output control signals.

[0077] In the above-mentioned input-output communication process between nodes, the output information of the nodes is transmitted in a vectorized manner, so gradients can be transmitted. Therefore, the above-mentioned multiple agents can be regarded as an agent as a whole to perform end-to-end gradient backpropagation. In the process of overall information transmission, noise can also be added to the information transmitted between nodes during training and inference, which helps to improve the multimodality (diversity of distribution) of the agent in processing the mapping between input and output.

[0078] Furthermore, training data is first collected when training each agent. Data sources include: expert demonstration data, where the demonstration duration covers one or more complete tasks, or a series of tasks; and automated policy data of the agents, using the collected data as raw input (including images, speech, and text). (representation) and raw output (including voice, text, actuator control signals, etc.) The data is a time-series record (represented by the machine name), and includes a variety of heterogeneous machine types, such as wheeled, wheel-footed, quadrupedal, and bipedal, as well as a variety of tasks, such as gripper grasping, dexterous hand grasping, navigation and object finding, etc.

[0079] Based on the collected data, an expert or automated evaluation mechanism is introduced to score the recorded time-series inputs and outputs. Negative feedback is given if there are errors or the target is not achieved; positive feedback is given if the execution is correct or the target is achieved; otherwise, zero feedback is given. The resulting demonstration data sequence is as follows:

[0080] ,

[0081] Where the superscript k represents the k-th trajectory, and T represents the length of the collected sequence. If the quality of the trajectories is confirmed to meet expectations, the sequence can be left unevaluated, and only the input and output sequences are retained, as shown in the following formula:

[0082] ,

[0083] If the collected data consists of demonstration and evaluation data, reinforcement learning will be the primary method for optimization. If the collected data consists of expert demonstration data, imitation learning will be used for optimization.

[0084] Furthermore, by utilizing massive amounts of corpora, images, videos, and multimodal alignment data, the central agent is pre-trained, the node neural network is initialized, and the parameters of all node neural networks are set to be shared or not shared. For each sequence, the memory tensors of all agents are initialized to 0 or random, and the input and output of each agent at each time step are calculated to obtain the output prediction of each actuator or terminal.

[0085] Specifically, if imitation learning is used, the loss function is as follows:

[0086]

[0087] in, This represents the loss value of imitation learning. This represents the input index of the actual motion data. Indicates time, Indicates the output prediction. The output data represents the actual motion data.

[0088] Specifically, if reinforcement learning is used, the loss function is as follows:

[0089] The specific formula for calculating the loss value based on the output prediction is as follows:

[0090]

[0091] in, This represents the loss value in reinforcement learning.

[0092] Specifically, the operation data of each agent at each time step is obtained through the information gradient transfer between agents. The output prediction is obtained based on the operation data of each agent at each time step. The loss value is calculated based on the output prediction and the evaluation data to train the agents until they converge, thus obtaining the trained agents. The agents are then deployed to the robot to control the robot's operation.

[0093] The gradient is transmitted from the terminal output to each node through inter-node communication and finally propagates to each agent. Pre-trained agents are used to perform warm start on some agents, such as the central agent, which can be warm started using multimodal pre-trained agents.

[0094] Through the above process, this embodiment of the invention achieves a significant improvement in the flexibility and intelligence of robot behavior through an adaptive swarm intelligence robot control method. By constructing intelligence agents for each joint of the robot and configuring connection relationships, as well as constructing a central intelligence agent to process diverse input information, the robot's environmental adaptability and decision-making efficiency are enhanced. The parameter sharing and synchronous update mechanism ensures collaborative work among intelligence agents. Combined with imitation learning and reinforcement learning, the robot can perform self-optimization based on real motion data and evaluation data, improving control accuracy and response speed. This provides an efficient and intelligent solution for robot applications in complex environments, effectively solving the problems of existing technologies being unable to achieve multi-task and multi-scenario robot control, and having poor generalization ability and flexibility.

[0095] The following are embodiments of the apparatus of the present invention, which can be used to execute the robot control method based on adaptive swarming intelligent agents involved in the present invention. For details not disclosed in the embodiments of the apparatus of the present invention, please refer to the method embodiments of the robot control method based on adaptive swarming intelligent agents involved in the present invention.

[0096] Please see Figure 3 This invention provides a robot control device 800.

[0097] The device 800 includes, but is not limited to: a joint agent setting module 810, a central agent construction module 830, a parameter mechanism setting module 850, and a robot training module 870.

[0098] The joint agent setting module 810 is used to build joint agents for each joint of the robot and configure the connection relationship between each joint agent; the joint agents include arm agents and waist agents.

[0099] The central agent construction module 830 is used to build a central agent for the robot and process input information through the central agent; the input information includes image signals, text signals and voice signals.

[0100] The parameter mechanism setting module 850 is used to set up a mechanism for parameter sharing and synchronous updating for each joint agent and the central agent; the central agent is used to output signals to control each joint agent.

[0101] The robot training module 870 is used to acquire real motion data and evaluation data. Based on the real motion data and evaluation data, the robot is controlled by training and optimizing the joint agents and the central agent through imitation learning and reinforcement learning.

[0102] It should be noted that the robot control provided in the above embodiments is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed. That is, the internal structure of the robot control device will be divided into different functional modules to complete all or part of the functions described above.

[0103] Furthermore, the robot control device provided in the above embodiments and the robot control method based on adaptive swarm intelligence belong to the same concept, and the specific way in which each module performs operations has been described in detail in the method embodiments, and will not be repeated here.

[0104] Compared with related technologies, the beneficial effects of the present invention are:

[0105] 1. This invention first constructs joint agents for each joint of the robot and configures the connection relationships between them, then constructs a central agent for the robot. The central agent processes input information, including image signals, text signals, and voice signals. A mechanism for parameter sharing and synchronous updating is set up for each joint agent and the central agent. The central agent outputs signals to control each joint agent, acquiring real motion data and evaluation data. Based on the real motion data and evaluation data, the joint agents and the central agent are trained and optimized through imitation learning and reinforcement learning before robot control is performed. This invention utilizes an adaptive swarm agent robot control method to achieve… This technology significantly enhances the flexibility and intelligence of robot behavior. By constructing intelligent agents for each joint of the robot and configuring their connections, as well as building a central intelligent agent to process diverse input information, it improves the robot's environmental adaptability and decision-making efficiency. The parameter sharing and synchronous update mechanism ensures collaborative work among intelligent agents. Combined with imitation learning and reinforcement learning, the robot can perform self-optimization based on real motion data and evaluation data, improving control accuracy and response speed. This provides an efficient and intelligent solution for robot applications in complex environments, effectively addressing the problems of existing technologies being unable to achieve multi-task and multi-scenario robot control, as well as poor generalization ability and flexibility.

[0106] 2. This invention can control heterogeneous robots, adapt to complex environments and tasks, and is an interactive robot intelligent agent base. This invention can achieve unified control of heterogeneous robots through distributed joint control, and can adapt to robots of any structure (wheeled, bipedal, quadrupedal, single-arm, dual-arm, gripper, dexterous hand, special robot) through intelligent agent parameters and the memory ability of each joint.

[0107] 3. This invention can utilize the memory capabilities of each joint agent and the general knowledge capabilities of the central agent to enable a set of agents to adapt to different tasks and characteristics, thereby achieving self-learning and adaptive control after deployment.

[0108] 4. This invention achieves implicit end-to-end optimization of input and output between joints through end-to-end optimization, eliminating the need for manually defining communication protocols and formats between various structures. At the same time, it retains the efficient and accurate information transmission of dense vectors, enabling more effective and precise inter-module collaboration and avoiding error propagation. It also allows asynchronous control between joints, such as slower central reaction time and shorter control cycles of the underlying intelligent agents, thereby achieving more flexible responses.

[0109] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0110] The above description is only a partial embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A robot control method based on adaptive swarm intelligence agents, characterized by, The method comprises: constructing joint agents for each joint of the robot and configuring connection relationship between the joint agents; the joint agents comprise arm agents and waist agents; constructing a central agent for the robot, and processing input information through the central agent; the input information comprises image signals, text signals and voice signals; setting a parameter sharing and synchronous updating mechanism for the joint agents and the central agent; the central agent is used for output signal control of the joint agents; obtaining real motion data and evaluation data, training and optimizing the joint agents and the central agent through imitative learning and reinforcement learning according to the real motion data and the evaluation data, and then performing robot control; introducing an expert evaluation mechanism to automatically evaluate the agents during the training process, and optimizing the parameters of the agents by reducing the error between the expert demonstration actions and the output of the agents, and training the agents through imitative learning; introducing an automatic evaluation mechanism to score the input and output recorded in the agents during the training process, and optimizing the agents through reinforcement learning according to the scoring results, and training the agents through reinforcement learning; the optimization of the parameters of the agents by reducing the error between the expert demonstration actions and the output of the agents comprises: inputting the input information recorded in the data of the expert demonstration actions into the agents to run, obtaining output prediction according to the running data of the agents at each time, and calculating loss value according to the output prediction and the output information recorded in the expert demonstration to train; the specific formula for calculating the loss value is as follows: wherein, denotes a loss value of imitation learning, denotes an input index of the expert demonstration data, denotes a time instant, denotes an output prediction of the agent, denotes an output data of the expert demonstration; the optimization of the agents through reinforcement learning according to the scoring results comprises: inputting the input value and the output value recorded in the agents into an automatic evaluation system to obtain an evaluation value, and inputting the input value recorded in the agents into the agents to obtain output prediction; calculating loss value according to the output prediction, the recorded output and the evaluation value to train; the specific formula for calculating the loss value is as follows: wherein, represents a loss value of reinforcement learning, is a decay coefficient, which is taken between 0.95 and 0.

99.

2. The robot control method based on adaptive swarm intelligence agents according to claim 1, wherein, the arm agent connects finger root agents and finger joint agents, and the waist agent connects hip joint agents, knee joint agents and ankle joint agents.

3. The robot control method based on adaptive swarm intelligence agents according to claim 2, wherein, The formula for information transmission between the interconnected agents is as follows: ； wherein, represents a parameter of the th agent, represents a memory tensor at the th time instant, represents an agent output at the th time instant, represents a structure of the th agent, represents an input at the th time instant.

4. The robot control method based on adaptive swarm intelligence agent according to claim 1, wherein, the construction of the central agent for the robot and the processing of input information through the central agent comprises: if the input information is a text signal, signal processing is performed on the text signal to obtain a symbol sequence, and the symbol sequence is converted into a vector sequence through the embedding coding layer of the central agent; if the input information is an image signal, the image signal is encoded into a vector sequence through a convolutional neural network; if the input information is a voice signal, the voice signal is processed into a vector sequence through a voice processing algorithm and a sequence encoding network.

5. The adaptive swarm of intelligent agents based robot control method of claim 1, wherein, the obtaining of real motion data and evaluation data, and the training and optimization of the joint agents and the central agent through imitative learning and reinforcement learning according to the real motion data and the evaluation data comprise: Obtaining real motion data and evaluation data of robots of different types performing tasks; the types include wheeled, wheeled-foot, quadruped, biped; the tasks include gripper grasping, dexterous hand grasping and navigation searching; Training the joint agents and the central agent according to the real motion data through an imitation learning algorithm; Training the joint agents and the central agent according to the real motion data and the evaluation data through a reinforcement learning algorithm.

6. A method for training a robot based on adaptive swarm intelligence agents as claimed in claim 1, wherein, The method comprises: Collecting training data; the training data includes expert demonstration data and automated strategy data; the expert demonstration data is data generated by experts demonstrating performing tasks; the automated strategy data is data generated by agents performing; 7. The robot training method based on adaptive swarm intelligence agents as claimed in claim 6, wherein, The method further comprises: Obtaining a central training set, initializing node neural networks in the central agent, and setting a sharing attribute of the node neural network parameters; the central training set includes a large amount of corpus, images, videos and multi-modal alignment data; When the loss value is optimized, the gradient is transmitted through back propagation and gradient transmission through communication between nodes so that all agent parameters are optimized; Using a multi-modal pre-training model to hot start the central agent, pre-training the central agent using the central training set, and optimizing the parameters of the central agent according to the input and output of the central agent at each time.

8. A robot control device based on adaptive swarm intelligence agents, characterized by, The device is used to implement the robot control method based on adaptive swarm intelligent agents according to claim 1, and the device comprises: A joint agent construction module for constructing joint agents for each joint of the robot and configuring connection relationships between the joint agents; the joint agents include arm agents and waist agents; A central agent construction module for constructing a central agent for the robot and processing input information through the central agent; the input information includes image signals, text signals and voice signals; A parameter mechanism setting module for setting a mechanism of parameter sharing and synchronous updating for the joint agents and the central agent; the central agent is used to output signals to control the joint agents; A robot training module for obtaining real motion data and evaluation data, training and optimizing the joint agents and the central agent through imitation learning and reinforcement learning according to the real motion data and the evaluation data, and then controlling the robot.

Images