Robot control method and device based on Kupman model and robot

Abstract

The invention provides a robot control method and device based on a Kupman model and a robot. The method is applied to a controller (provided with a Kupman model and a control model) in a robot. The method comprises the steps that first motion state data and a control target of the robot at the current moment are acquired; the first motion state data are input into a Kupman model, second motion state data of the robot at the next moment are obtained, values of all parameters in the Kupman model are determined according to operation results of the robot in multiple different operation environments, the operation environments comprise non-simulation operation environments, all the parameters comprise target parameters, and the target parameters comprise target parameters; the importance degree of each target parameter in each operation environment is greater than a preset degree; inputting the first motion state data, the second motion state data and the control target into a control model to obtain a control instruction of the robot at the next moment; and executing the control instruction. The control precision of the robot can be remarkably improved.

Description

Technical Field

[0001] The present application relates to the field of robotics technology, and in particular to a robot control method, device, and robot based on a Koopman model. Background Art

[0002] When training robot control models such as reinforcement learning and model predictive control, algorithm design and model optimization are typically performed under idealized conditions, such as using a physical simulator or training the control model in a predefined uncertainty environment. However, in actual deployments, environmental conditions are often complex and variable, with various uncertainties (such as friction, noise, and stiffness variations). Directly transferring a trained control model to the robot's controller often leads to performance degradation or even failure. While existing methods such as meta-learning, policy fine-tuning, and domain randomization can alleviate this problem to a certain extent, their effectiveness in improving robot control accuracy is limited, and they still cannot effectively achieve robot control in complex and changing environmental conditions. Summary of the Invention

[0003] The present application provides a robot control method, device and robot based on the Koopman model, which are used to solve the defects of the existing technology in improving the control accuracy of the robot, and can achieve precise control of the robot.

[0004] The present application provides a robot control method based on the Koopman model, which is applied to a controller in the robot. The Koopman model and a control model are deployed in the controller. The method includes: Obtaining first motion state data and control target of the robot at a current moment; Inputting the first motion state data into the Koopman model to predict second motion state data of the robot at a next moment, wherein values ​​of various parameters in the Koopman model are determined based on operating results of the robot in a plurality of different operating environments, the operating environments including non-simulation operating environments, the various parameters including target parameters, and importance of each target parameter in each of the operating environments being greater than a preset value; Inputting the first motion state data, the second motion state data, and the control target into the control model to obtain a control instruction for the robot at the next moment; Control the robot to execute the control instruction.

[0005] According to the robot control method based on the Koopman model provided in the present application, the Koopman model includes an encoding module and an operator module. The first motion state data is input into the Koopman model to predict the second motion state data of the robot at the next moment, including: Inputting the first motion state data into the encoding module to obtain a state representation of the first motion state data in a target feature space, wherein the dimension of the target feature space is higher than the dimension of an original state space in which the first motion state data is located; The state representation is input into the operator module to obtain the second motion state data. The operator module is used to perform a linear transformation on the state representation through a Koopman matrix and obtain the second motion state data after the linear transformation.

[0006] According to the robot control method based on the Koopman model provided in the present application, the first motion state data is input into the target Koopman model to predict the second motion state data of the robot at the next moment, including: Inputting the first motion state data into a target Koopman model to predict the second motion state data of the robot at the next moment; The target Koopman model is obtained by adjusting the values ​​of target parameters in the Koopman model according to the operation results of the robot in the current working environment, and the adjusted Koopman model is the target Koopman model.

[0007] According to the robot control method based on the Koopman model provided in this application, the target parameters in the Koopman model are determined by the following steps: Obtaining the operation results of the robot in each of the operation environments; According to each of the operation results, a Koopman operator model corresponding to each of the operation environments is trained, and the network structure of each of the Koopman operator models is the same; Determining the importance score of each parameter in each of the Koopman operator models through an elastic weight solidification algorithm; According to the importance score, a target parameter in the Koopman model is determined.

[0008] According to the robot control method based on the Koopman model provided in this application, the elastic weight solidification algorithm is used to determine the importance score of each parameter in each Koopman operator model, including: Determining the Fisher information matrix corresponding to each of the Koopman operator models; The importance score of each parameter in each Koopman operator model is determined according to the diagonal elements in the Fisher information matrix.

[0009] According to the robot control method based on the Koopman model provided in the present application, the Koopman model has the same network structure as each of the Koopman operator models, and determining the target parameters in the Koopman model according to the importance score includes: For each parameter in the Koopman model, determining a comprehensive importance score according to the importance score of the corresponding parameter in each Koopman operator model; The parameter whose comprehensive importance score in the Koopman model is greater than the importance score threshold is determined as the target parameter.

[0010] According to the robot control method based on the Koopman model provided in the present application, the value of each parameter in the Koopman model is determined based on the average value of the values ​​of the corresponding parameters in each of the Koopman operator models, or is determined based on the value of the corresponding parameter in the Koopman operator model corresponding to an operating environment similar to the current working environment of the robot.

[0011] According to the robot control method based on the Koopman model provided in this application, the control model is obtained by the following steps: Constructing a digital model of the robot and a virtual working environment in a simulator; determining a training task for the robot based on the digital model and the virtual working environment; An initial control model of the robot is obtained, and the initial control model is trained according to the training task until the initial control model meets a preset condition, and the initial control model that meets the preset condition is the control model.

[0012] The present application also provides a robot control device based on the Koopman model, which is applied to a controller in a robot. The Koopman model and the control model are deployed in the controller. The device includes: A first acquisition module is used to obtain the first motion state data and control target of the robot at a current moment; a prediction module, configured to input the first motion state data into the Koopman model to predict second motion state data of the robot at a next moment, wherein values ​​of parameters in the Koopman model are determined based on operating results of the robot in a plurality of different operating environments, the operating environments including non-simulation operating environments, the parameters including target parameters, and the importance of each target parameter in each of the operating environments being greater than a preset degree; a second acquisition module, configured to input the first motion state data, the second motion state data, and the control target into the control model to obtain a control instruction for the robot at a next moment; A control module is used to control the robot to execute the control instruction.

[0013] The present application also provides a robot, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the robot control method based on the Koopman model as described above is implemented.

[0014] The present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the robot control method based on the Koopman model as described above is implemented.

[0015] The present application also provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the above-described Koopman model-based robot control methods.

[0016] In this application, the control model and the Koopman model are combined to control the robot, which has at least the following effects: First, the Koopman model can be trained based on operational results in representative non-simulation environments (e.g., real-world work environments). Compared to models trained under single or idealized conditions, it can more accurately capture the complex dynamic characteristics of robots in the real world. Specifically, by providing a more realistic system dynamics model (the Koopman model itself is designed to achieve approximate linear modeling), the Koopman model can more accurately predict the robot's motion state data at the next moment (i.e., the second motion state data). This enables the control model to obtain accurate control instructions based on this more precise second motion state data, significantly improving the robot's control accuracy.

[0017] Second, it improves the model's generalization and robustness. Because the Koopman model can be trained based on operational results in representative non-simulation environments, it has learned and internalized a variety of real-world dynamic characteristics. Compared to models trained only in a single idealized environment (such as pure simulation) or a single specific real environment, it has better adaptability and greater robustness to new environments that it has not fully seen but may have similar characteristics to the training environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0019] Figure 1 This is a flow chart of a robot control method based on the Koopman model according to an embodiment of the present application; Figure 2 This is a robot control device based on the Koopman model shown in one embodiment of the present application; Figure 3 This is a schematic diagram of the physical structure of a robot according to an example embodiment of the present application. DETAILED DESCRIPTION

[0020] To make the objectives, technical solutions, and advantages of this application more clear, the technical solutions in this application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0021] When developing robotic control algorithms (for tasks like grasping objects), researchers typically first build a virtual robot and environment in a computer—a physical simulator. In this simulator, all physical parameters are known and ideal. For example, there's no or constant friction between parts, precise motor force output, no noise interference from sensors, and the object's weight, shape, and position are perfectly aligned. For example, to train a robotic arm to precisely grasp a wooden block of a specific shape and weight from a table and place it in a specified location, idealized training involves creating a perfect virtual robotic arm, a virtual block, and a virtual table in the simulation software. The block's weight is set to a precise 100 grams, its surface is perfectly smooth, each joint of the robotic arm is frictionless, the motors react precisely, and the camera captures crisp, clear images. The control algorithm is then repeatedly trained in this clean environment until it perfectly performs the task.

[0022] However, after deploying the control model trained in simulation to a real robotic arm, the friction in the real joints requires more force for the robotic arm to move to the predetermined position, resulting in slow movement or inaccurate positioning. If the wooden block is slightly heavier or lighter than set in the simulation, or the surface is slippery, the robotic arm will grasp it unsteadily, or damage the wooden block by applying too much force. The camera may not be able to accurately locate the wooden block due to lighting changes or shadows. If the real environment differs greatly from the simulated environment, such as a large deviation in the actual position of the wooden block, or a joint of the robotic arm is slow to respond due to increased friction, the control algorithm may be completely unable to locate the wooden block, or it may collide or fall when trying to grasp it, resulting in mission failure.

[0023] In order to address this gap between ideal and reality, some methods have been proposed in the prior art: Meta-Learning: This allows robots to learn how to learn. By training on a large number of different simulation tasks (such as grasping tasks with different friction coefficients and object weights), the model can quickly adapt to new tasks.

[0024] Policy fine-tuning: Using the model trained in simulation as a basis, fine-tuning it on a real robot using a small amount of real data to adapt it to the real environment.

[0025] Domain Randomization: During simulation training, a large amount of random variation is intentionally introduced, such as randomly changing the color, size, friction coefficient, lighting conditions, etc. The goal is to train the model on a very diverse set of simulated data, making it less sensitive to unknown changes in the real world and more robust.

[0026] However, the above methods have very limited effects in improving the control accuracy of the robot, and still cannot achieve precise control of the robot when facing complex and changing environmental conditions.

[0027] The robot control method based on the Koopman model of the present application is applied to a controller in the robot, and the Koopman model and the control model are deployed in the controller. Figure 1 This is a flow chart of a robot control method based on the Koopman model shown in one embodiment of the present application. Figure 1 , the method of the present application may include the following steps: Step 101: Obtain the first motion state data and control target of the robot at the current moment.

[0028] In this application, the robot's motion state data is used to describe the robot's state in multiple dimensions (configured based on actual needs). Taking a robotic arm as an example, the robotic arm's first motion state data at the current moment may include joint angles, joint velocities, end effector positions, and sensor readings.

[0029] A control objective is a given goal that a robot is expected to achieve. For example, if the robot is a robotic arm, a control objective might be to precisely move the end effector of the robotic arm to a specific target point in the three-dimensional workspace and achieve a predetermined posture.

[0030] Step 102: Input the first motion state data into the Koopman model to predict the second motion state data of the robot at the next moment. The values ​​of various parameters in the Koopman model are determined based on the operating results of the robot in multiple different operating environments, including non-simulation operating environments. The various parameters include target parameters, and the importance of each target parameter in each operating environment is greater than a preset degree.

[0031] In this application, the Koopman model is used to accurately predict the robot's motion state at the next moment.

[0032] In specific implementations, the robot's operating results from multiple different operating environments are collected in advance. These results are then analyzed to identify the parameters that are most important in each operating environment. These parameters are then used as target parameters (key parameters). The Koopman model includes all of these target parameters. Furthermore, the value of each parameter in the Koopman model can be determined based on a comprehensive analysis of multiple operating results. The specific method for determining this value will be described in detail later.

[0033] The operating environment of this application may include non-simulated (non-virtual) operating environments. In actual implementation, these operating environments may all be representative real working environments, or a combination of real working environments and environments with uncertain changing patterns. Of course, the operating environment can also be selected based on actual circumstances.

[0034] Step 103: Input the first motion state data, the second motion state data, and the control target into the control model to obtain the control instruction of the robot at the next moment.

[0035] In this application, the control model is a model pre-trained in a simulator. The control model comprehensively analyzes the first motion state data, the second motion state data, and the control target to obtain the control instruction of the robot at the next moment.

[0036] The present application does not impose any specific restrictions on the type of control model, and it may be, for example, a PID control model, a Model Predictive Control (MPC) model, etc.

[0037] Step 104: Control the robot to execute the control instruction.

[0038] In this application, after obtaining a control instruction, the robot is controlled to execute the control instruction at a corresponding moment to achieve a given control target.

[0039] In this application, the control model and the Koopman model are combined to control the robot, which has at least the following effects: First, the Koopman model can be trained based on operational results in representative non-simulation environments (e.g., real-world work environments). Compared to models trained under single or idealized conditions, it can more accurately capture the complex dynamic characteristics of robots in the real world. Specifically, by providing a more realistic system dynamics model (the Koopman model itself is designed to achieve approximate linear modeling), the Koopman model can more accurately predict the robot's motion state data at the next moment (i.e., the second motion state data). This enables the control model to obtain accurate control instructions based on this more precise second motion state data, significantly improving the robot's control accuracy.

[0040] Second, the model's generalization and robustness are improved. Because the Koopman model can be trained based on operational results in representative non-simulation environments, it has already learned and internalized a variety of real-world dynamic characteristics. Compared to models trained only in a single idealized environment (such as pure simulation) or a single specific real-world environment, it has better adaptability and robustness to new environments that have not been fully seen but may have similar characteristics to the training environment, and its dynamic representation is more universal.

[0041] In combination with the above embodiments, in one implementation, the first motion state data is input into the target Koopman model to predict the second motion state data of the robot at the next moment, including: The first motion state data is input into a target Koopman model to predict the second motion state data of the robot at the next moment; wherein the target Koopman model is obtained by adjusting the values ​​of target parameters in the Koopman model according to the operating results of the robot in the current working environment, and the adjusted Koopman model is the target Koopman model.

[0042] In this application, after the trained Koopman model is deployed to the robot's controller for the first time, the robot can be allowed to run in its current working environment for a period of time, and the robot's operating results during this period can be collected. Then, based on the operating results, the values ​​of the target parameters in the Koopman model can be adjusted to obtain the adjusted Koopman model (i.e., the target Koopman model).

[0043] Among them, when adjusting the value of each target parameter, it is necessary to keep the remaining parameters in the Koopman model in a frozen state (that is, the corresponding parameter values ​​remain unchanged), that is, only the value of the target parameter is fine-tuned, while the values ​​of the remaining parameters remain unchanged. In this way, only a small amount of operating results of the robot in the current working environment need to be collected to achieve the ideal control accuracy. In other words, if a large number of operating results of the robot in the current working environment are used to fine-tune the value of the target parameter, the control accuracy can be further improved (if the number of collected operating results of the robot in the current working environment is the same as the number of collected operating results of the robot in the current working environment in the prior art, the control accuracy of the robot in this application will be higher).

[0044] In combination with the above embodiments, in one implementation, the Koopman model includes an encoding module and an operator module, and the first motion state data is input into the Koopman model to predict the second motion state data of the robot at the next moment, including: Inputting the first motion state data into an encoding module to obtain a state representation of the first motion state data in a target feature space, where the dimension of the target feature space is higher than the dimension of an original state space where the first motion state data is located; The state representation is input into the operator module to obtain second motion state data. The operator module is used to perform a linear transformation on the state representation through the Koopman matrix and obtain the second motion state data after the linear transformation.

[0045] In this application, the Koopman model includes an encoder module and an operator module. The encoding module is the encoder module, and the operator module is the operator module.

[0046] The encoder module is used to map the first motion state data to the target feature space to obtain a state representation of the first motion state data in the target feature space. The operator module is used to perform a linear transformation on the state representation using the Koopman matrix to obtain the second motion state data after the linear transformation.

[0047] Through the encoding module and operator module in the Koopman model, the present application can accurately predict the robot's motion state data at the next moment, thereby improving the accuracy of the control instructions subsequently obtained.

[0048] In combination with the above embodiments, in one embodiment, the target parameters in the Koopman model are determined by the following steps: Step 1: Obtain the robot's operating results in various operating environments.

[0049] Step 2: Based on the running results, the Koopman operator model corresponding to each running environment is trained. The network structure of each Koopman operator model is the same.

[0050] For example, based on the operating results obtained by the robot in operating environment 1, the Koopman operator model 1 corresponding to operating environment 1 is trained. Based on the operating results obtained by the robot in operating environment 2, the Koopman operator model 2 corresponding to operating environment 2 is trained. And so on, the Koopman operator model corresponding to each operating environment is obtained.

[0051] Step 3: Determine the importance score of each parameter in each Koopman operator model through the elastic weight solidification algorithm.

[0052] For example, the Koopman operator model includes Koopman operator models 1-10. After step 3, the importance scores of each parameter in Koopman operator model 1, the importance scores of each parameter in Koopman operator model 2, ..., and the importance scores of each parameter in Koopman operator model 10 can be obtained.

[0053] Specifically, step 3 can be implemented as follows: Determine the Fisher information matrix corresponding to each Koopman operator model; The importance scores of each parameter in each Koopman operator model are determined based on the diagonal elements in the Fisher information matrix.

[0054] Of course, in actual implementation, the importance score of each parameter in each Koopman operator model can also be determined by other methods.

[0055] Step 4: According to the importance scores, the target parameters in the Koopman model are determined. The network structure of the Koopman model is the same as that of each Koopman operator model.

[0056] In this application, the network structure of the Koopman model and each Koopman operator model is the same (both include an encoding module and an operator module), that is, the Koopman model and each Koopman operator model include the same parameters, but the values ​​of the parameters are different.

[0057] In actual implementation, step 4 can be achieved as follows: For each parameter in the Koopman model, the corresponding comprehensive importance score is determined according to the importance score of the corresponding parameter in each Koopman operator model; The parameters whose comprehensive importance scores in the Koopman model are greater than the importance score threshold are determined as target parameters.

[0058] The following describes in detail the process of obtaining the Koopman model in this application through the first and second stages.

[0059] Phase 1: Multi-environment modeling.

[0060] The robot collects operational data from multiple representative real-world environments (typical environments the robot might encounter in the future) or environments with uncertain dynamics (environments with factors that are difficult to accurately model or that vary randomly). Each environment is treated as an independent task and a corresponding Koopman operator model is trained. Each Koopman operator model consists of an encoder module and an operator module. These modules approximate linear modeling of the system dynamics.

[0061] In each of the above-selected operating environments, the robot is controlled to perform a series of actions, and its motion state information (i.e., the first motion state information) and corresponding control instructions are recorded as the basis for subsequent modeling.

[0062] The core idea of ​​the Koopman operator theory is that even if the behavior in the original state space of a dynamic system is nonlinear, there may be a high-dimensional observation function space in which the evolution of the system can be described by a linear operator (Koopman operator). The Encoder (operator) module is a module that maps (converts or elevates) the robot's original, possibly high-dimensional and nonlinear motion state data (such as a series of readings from sensors) to a new, usually high-dimensional feature space (i.e., the above-mentioned observation function space). This new space is chosen to make the system dynamics exhibit as linear behavior as possible in this space. The Encoder module is usually implemented using a neural network. The Operator (operator) module (usually a linear matrix, i.e., the Koopman matrix) is responsible for predicting the evolution of the elevated state at the next time step. Ideally, ,in is the original state, corresponding to the first motion state data input, is the encoder function, corresponding to the operation in the Encoder module, and K is the Koopman matrix in the Operator module.

[0063] In this application, a separate set of encoder and operator parameters is trained for each runtime environment based on the collected runtime results. For example, runtime environment A corresponds to the Koopman model (Encoder_A, Operator_A), runtime environment B corresponds to the Koopman model (Encoder_B, Operator_B), and so on.

[0064] Therefore, although the complete dynamics model of a robot in the real world is extremely complex and highly nonlinear, after the encoder maps the state into a suitable lift space, its dynamic evolution can be approximated using a relatively simple linear operator. This linearization greatly simplifies subsequent analysis, prediction, and control tasks.

[0065] For example, consider a six-axis robotic arm whose task is to grasp and place objects under different conditions. The steps to obtain the Koopman operator models corresponding to multiple different operating environments are as follows: STEP 1: Select multiple representative real-world working environments or environments with uncertain changing patterns.

[0066] Environment 1 (R1): Operating on a standard workbench with a known and fixed friction coefficient of the table, good lighting conditions, and grasping objects of fixed weight (e.g., 1 kg).

[0067] Environment 2 (R2): Operating on a slightly tilted table (e.g., 5 degrees), with a different tabletop friction coefficient than R1, stable but dim lighting conditions, and grasping the same 1 kg object.

[0068] Environment 3 (R3): Operating on a standard workbench, but with some natural wear of the lubrication at the end joint of the robotic arm, resulting in increased friction at the joint (a form of uncertainty), while grasping the same 1kg object.

[0069] Environment 4 (R4): Operates on a standard workbench, but this time the weight of the object to be grasped is changed to 1.5 kg (another uncertainty, load variation).

[0070] STEP 2: Data collection.

[0071] For environment 1 (R1), the robot arm is controlled to perform multiple grasping and placing tasks in this environment. During the tasks, the following data are collected: (1) Motion state data: the angle, angular velocity, and angular acceleration of each joint; the position and posture of the end effector; the motor current; and other readings from the torque sensor or vision sensor. (2) Control input data: the control instructions sent to each joint motor (such as the target angle, velocity, or torque). This data is stored to form data set D1. Similarly, the robot arm is operated in environment 2 (R2) to collect data set D2; the robot arm is operated in environment 3 (R3) to collect data set D3. The robot arm is operated in environment 4 (R4) to collect data set D4.

[0072] STEP 3: Train the Koopman operator model.

[0073] Task 1 (for environment R1): Using dataset D1, design a neural network as Encoder_1. The input is the motion state of the robot in R1. , the output is the state after promotion . At the same time, learn an Operator_1 (usually a matrix K1), and the training goal is to make As close to K1 as possible After training is completed, the model M1={Encoder1,Operator1} is obtained.

[0074] Task 2 (for environment R2): Using dataset D2, repeat the above process to train the model M2 = {Encoder2, Operator2}. Due to different environment dynamics (tilt, different friction), the parameters of Encoder2 and Operator2 will be different from those in M1.

[0075] Task 3 (for environment R3): Using dataset D3, repeat the above process to train the model M3={Encoder3,Operator3}.

[0076] Task 4 (for environment R4): Using dataset D4, repeat the above process to train the model M4={Encoder4,Operator4}.

[0077] Through the above steps, multiple Koopman operator models (M1, M2, M3, M4) can be obtained.

[0078] In this application, the training method of a single Koopman operator model can be implemented by referring to the training method in the existing technology, and this application does not impose any restrictions on this.

[0079] Phase 2: Parameter importance assessment.

[0080] In this application, the Elastic Weight Consolidation (EWC) algorithm in lifelong learning is used to evaluate the importance of encoder parameters and operator parameters in multiple different Koopman operator models, and identify important parameters that have a common critical impact in multi-task scenarios, namely target parameters.

[0081] The core idea of ​​the EWC algorithm is that after a model learns a task, it estimates the importance of each model parameter for that task. When learning the next task, EWC adds a penalty term to the loss function. This penalty term limits large changes to parameters that were important for the previous task. Parameter importance is typically approximated by the diagonal elements of the Fisher Information Matrix (FIM). Parameters with greater Fisher Information have a greater impact on the model's output (performance on a specific task) and are therefore more important.

[0082] Continuing with the example of the six-axis robotic arm used in the previous article, assume that models for the following four real environments have been obtained: model M1 (Encoder1, Operator1), model M2 (Encoder2, Operator2), model M3 (Encoder3, Operator3), and model M4 (Encoder4, Operator4).

[0083] The encoders in different Koopman operator models have the same network structure, but the specific parameter values ​​are different. The operators in different Koopman operator models have the same network structure, but the specific parameter values ​​are different.

[0084] The steps for implementing parameter importance assessment are as follows: (1) Calculate the importance of each model parameter for its own task: For model M1: Compute the diagonal elements of the Fisher information matrix of all parameters in model M1 (parameters of Encoder1 and Operator1) using its training data D1. This diagonal vector Each element in represents the importance score of the corresponding parameter in M1 for performing well in environment R1. The larger the value, the more important it is.

[0085] Similarly, for model M2, use data D2 to calculate the importance score of its parameters and get .

[0086] For model M3, use data D3 to calculate the importance score of its parameters and get .

[0087] For model M4, use data D4 to calculate the importance score of its parameters and get .

[0088] (2) Aggregate important information and identify common key parameters: Based on four sets of importance scores , each group corresponds to the importance score of each parameter of a model in its specific environment. Then align the parameters. Since all models share the same network structure, These vectors have the same dimension, and each position corresponds to the parameter at the same position in the network structure. Next, calculate the importance score: parameters, and their comprehensive importance score This can be done by taking the average value (i.e. ) or take the weighted average.

[0089] Through the above steps, we can get a comprehensive score vector of the importance of each parameter in the general model architecture. .

[0090] (3) Determine the importance parameter (target parameter): Based on the overall importance score , set an importance score threshold.

[0091] All parameters whose comprehensive importance scores are higher than the importance score threshold are determined as target parameters.

[0092] If the weight of a neuron in the network of the encoding module A very high value indicates that it is a critical parameter across multiple environments.

[0093] Through the above second stage, it is possible to determine which parameters are important parameters (target parameters) in the general Koopman model.

[0094] In the present application, the value of each parameter in the Koopman model can be determined according to the average value of the corresponding parameter in each Koopman operator model. For example, for parameter a in the Koopman model, it can be the average value of the value of parameter a in each Koopman operator model.

[0095] Secondly, the values ​​of the various parameters in the Koopman model can also be determined based on the values ​​of the corresponding parameters in the Koopman operator model corresponding to an operating environment similar to the robot's current operating environment. For example, if the operating environment similar to the robot's current operating environment is environment 1 (R1), then the value of any parameter b in the Koopman model can be the value of the corresponding parameter in the Koopman operator model corresponding to environment 1 (R1).

[0096] In this application, the values ​​of each parameter in the Koopman model are determined according to the values ​​of each parameter in multiple Koopman operator models, which enables the Koopman model to better adapt to various complex real environments, achieve accurate prediction of the robot's motion state, and thus significantly improve the robot's control effect.

[0097] In combination with the above embodiments, in one implementation, the control model is obtained through the following steps: Build a digital model of the robot and a virtual working environment in the simulator; Determine the robot's training tasks based on the digital model and virtual working environment; An initial control model of the robot is obtained, and the initial control model is trained according to the training task until the initial control model meets the preset conditions. The initial control model that meets the preset conditions is the control model.

[0098] The following describes the training process of the control model through the third stage: Phase 3: Training a basic control model in the simulator.

[0099] A simulator is a virtual environment that simulates the physical properties of a robot (shape, mass, joints, sensors, etc.) and its interactions with the virtual environment (collisions, forces, etc.).

[0100] (1) Constructing digital models of the simulation environment and robots: Import or create an accurate 3D model of the six-axis robotic arm in your chosen simulation software, including its link lengths, mass distribution, joint types, range of motion, motor parameters (e.g., maximum torque, speed limits), etc.

[0101] A standardized task scenario (virtual work environment) is set up. For example, a virtual wooden block of standard shape and weight (e.g., 0.8 kg) is grasped from fixed position A on a workbench and then placed at fixed position B. The friction coefficients of the workbench surface and the friction coefficients between the wooden block and the gripper are all set to ideal, known values ​​in the simulation.

[0102] (2) Select and construct the initial control model of the controller, assuming that the MPC model is adopted.

[0103] MPC-based controller design: Set the prediction time domain (e.g. the next N time steps) and control time domain of MPC.

[0104] Define a cost function to guide MPC optimization, for example: Minimize the distance between the end effector and the target point (A or B).

[0105] Minimize joint movement speed or acceleration for smoothness.

[0106] Minimize the magnitude of control inputs (such as motor torque) to save energy.

[0107] (3) Training the MPC-based controller in simulation: The simulated robotic arm with the MPC controller repeatedly performs pick-and-place tasks in a virtual work environment until the robotic arm can complete the task efficiently and accurately in the simulation environment.

[0108] The following will describe in detail the process of realizing robot control by combining the Koopman model and the control model in this application through the fourth stage.

[0109] Phase 4: In the actual application scenario, the Koopman model and control model are deployed in the robot controller. The control model is the MPC model trained in Phase 3.

[0110] After the deployment is completed, the values ​​of the target parameters in the Koopman model are first fine-tuned according to the robot's operating results in the current working environment to obtain the fine-tuned target Koopman model.

[0111] In this application, during the fine-tuning process, only the target parameters (i.e., the encoder / operator parameters with higher importance) are fine-tuned, and the remaining parameters remain frozen.

[0112] In this application, through the online learning mechanism, the Koopman model is fine-tuned using a small amount of data in the current task environment, which can enable the Koopman model to quickly adapt to environmental changes and achieve low-resource, high-efficiency adjustment of the control strategy.

[0113] While meta-learning, policy fine-tuning, and domain randomization employed in related technologies have shown some effectiveness, they still suffer from the following shortcomings: First, regarding sample efficiency, if methods such as policy fine-tuning are employed, a significant amount of real-world sample data is required, and collecting real-world robot data is typically costly and time-consuming. Second, regarding convergence speed, it can take a long time for the model to adapt to a new environment or learn from scratch. Third, regarding generalization capability, even with domain randomization, the model still cannot cope well with extreme situations or new uncertainties in the real environment that were completely unforeseen during simulation. In other words, the model's ability to generalize from training data to unseen data still needs to be improved.

[0114] For the first problem, this application only fine-tunes the target parameters in the Koopman model, while the other parameters remain unchanged, which greatly reduces the number of parameters that need to be adjusted. The model no longer needs massive data to adapt to the new environment, which can effectively solve the first problem.

[0115] Regarding the second problem, the importance assessment in the second stage ensures the preservation and streamlined migration of the model's key structures. In the fourth stage, by fine-tuning only a small number of target parameters, the online learning process's search space is reduced, allowing for faster convergence to an effective state adapted to the current environment, enabling efficient policy adjustments and effectively solving the second problem.

[0116] As for the third question, in the first phase of this application, the Koopman operator is used to implement multi-environment dynamics modeling, which improves the system's expressibility of complex dynamics in the real world. This enables the model itself to have a cognitive basis for a series of real dynamic changes, rather than just randomized simulation parameters. Secondly, the importance evaluation mechanism of the second phase helps to retain the key structures of the model. These key structures carry common knowledge learned from multiple real environments, which helps the model maintain a certain basic performance when encountering new situations. Furthermore, the online learning mechanism of the fourth phase allows the model to quickly adapt to environmental changes by fine-tuning the target parameters. Even if the new environment has characteristics that are not fully covered in the multi-environment modeling in the first phase, the model can adapt to these new situations by adjusting its most sensitive and relevant parts (ie, target parameters), thereby improving the robustness of the algorithm and the feasibility of its application.

[0117] In summary, the method of this application has at least the following technical effects: First, the Koopman model can be trained based on operational results in representative non-simulation environments (e.g., real-world work environments). Compared to models trained under single or idealized conditions, it can more accurately capture the complex dynamic characteristics of robots in the real world. Specifically, by providing a more realistic system dynamics model (the Koopman model itself is designed to achieve approximate linear modeling), the Koopman model can more accurately predict the robot's motion state data at the next moment (i.e., the second motion state data). This enables the control model to obtain accurate control instructions based on this more precise second motion state data, significantly improving the robot's control accuracy.

[0118] Second, it enhances the ability to represent complex dynamics. This application uses the Koopman operator theory to model the dynamic characteristics of robotic systems in a variety of different environments, significantly improving the accuracy and depth of the model in capturing and expressing complex, nonlinear dynamic behaviors.

[0119] Third, key knowledge retention and efficient transfer are achieved. This application introduces a parameter importance assessment mechanism (such as the elastic weight curing (EWC) algorithm) derived from the field of lifelong learning, which can accurately identify and protect the core parameter structure of the model formed during multi-environment training. This allows key learned knowledge to be retained when facing new environments or tasks, and can be transferred in a more streamlined and efficient manner, avoiding catastrophic forgetting and accelerating the adaptation process.

[0120] Fourth, it reduces actual deployment costs. This application can significantly reduce the amount of actual interaction data required for algorithm deployment and online adaptive adjustment on real robot systems while maintaining or even improving robot control accuracy, and significantly reduce the time and computing resource overhead required for online learning.

[0121] Fifth, it improves practical application performance and robustness. This application supports rapid and efficient online adjustment of control strategies in a changing real-world physical environment. This not only enhances the robustness of the control algorithm in the face of unforeseen environmental disturbances and system uncertainties, but also greatly improves the feasibility and practicality of the algorithm's successful deployment and stable operation in real-world industrial applications and complex scenarios.

[0122] The following describes a robot control device based on the Koopman model provided in the present application. The robot control device based on the Koopman model described below and the robot control method based on the Koopman model described above can refer to each other.

[0123] The robot control device based on the Koopman model provided in the present application is applied to a controller in the robot, in which the Koopman model and the control model are deployed. Figure 2 This is a robot control device based on the Koopman model shown in one embodiment of the present application. Figure 2 The robot control device 200 based on the Koopman model of the present application may include: A first acquisition module 201 is used to obtain the first motion state data and control target of the robot at the current moment; a prediction module 202 configured to input the first motion state data into the Koopman model to predict second motion state data of the robot at a next moment, wherein values ​​of parameters in the Koopman model are determined based on operating results of the robot in a plurality of different operating environments, the operating environments including non-simulation operating environments, the parameters including target parameters, and importance of each target parameter in each of the operating environments being greater than a preset value; A second acquisition module 203 is configured to input the first motion state data, the second motion state data, and the control target into the control model to obtain a control instruction for the robot at the next moment; The control module 204 is used to control the robot to execute the control instruction.

[0124] According to the robot control device 200 based on the Koopman model provided in this application, the Koopman model includes an encoding module and an operator module, and the prediction module 202 includes: a first input submodule, configured to input the first motion state data into the encoding module to obtain a state representation of the first motion state data in a target feature space, wherein the dimension of the target feature space is higher than the dimension of an original state space in which the first motion state data is located; The second input submodule is used to input the state representation into the operator module to obtain the second motion state data. The operator module is used to perform a linear transformation on the state representation through the Koopman matrix and obtain the second motion state data after the linear transformation.

[0125] According to the robot control device 200 based on the Koopman model provided in this application, the prediction module 202 includes: a third input submodule, configured to input the first motion state data into a target Koopman model to predict and obtain second motion state data of the robot at a next moment; The target Koopman model is obtained by adjusting the values ​​of target parameters in the Koopman model according to the operation results of the robot in the current working environment, and the adjusted Koopman model is the target Koopman model.

[0126] The robot control device 200 based on the Koopman model provided in this application further includes a determination module, which is used to implement: Obtaining the operation results of the robot in each of the operation environments; According to each of the operation results, a Koopman operator model corresponding to each of the operation environments is trained, and the network structure of each of the Koopman operator models is the same; Determining the importance score of each parameter in each of the Koopman operator models through an elastic weight solidification algorithm; According to the importance score, a target parameter in the Koopman model is determined.

[0127] According to the robot control device 200 based on the Koopman model provided in this application, the determination module is specifically used to: Determining the Fisher information matrix corresponding to each of the Koopman operator models; The importance score of each parameter in each Koopman operator model is determined according to the diagonal elements in the Fisher information matrix.

[0128] According to the robot control device 200 based on the Koopman model provided in this application, the Koopman model has the same network structure as each of the Koopman operator models, and the determination module is specifically used to: For each parameter in the Koopman model, determining a corresponding comprehensive importance score according to the importance score of the corresponding parameter in each Koopman operator model; The parameter whose comprehensive importance score in the Koopman model is greater than the importance score threshold is determined as the target parameter.

[0129] The robot control device 200 based on the Koopman model provided in this application further includes a training module for implementing: Constructing a digital model of the robot and a virtual working environment in a simulator; determining a training task for the robot based on the digital model and the virtual working environment; An initial control model of the robot is obtained, and the initial control model is trained according to the training task until the initial control model meets a preset condition, and the initial control model that meets the preset condition is the control model.

[0130] According to the Koopman model-based robot control device 200 provided in the present application, the values ​​of the various parameters in the Koopman model are determined based on the average values ​​of the corresponding parameters in the various Koopman operator models, or are determined based on the values ​​of the corresponding parameters in the Koopman operator model corresponding to an operating environment similar to the current working environment of the robot.

[0131] Figure 3 This is a schematic diagram of the physical structure of a robot according to an embodiment of the present application. Figure 3 As shown, the robot may include: a processor 310, a communication interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communication interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 may call logic instructions in the memory 330 to execute a robot control method based on a Koopman model, the method comprising: obtaining first motion state data and a control target of the robot at a current moment; inputting the first motion state data into the Koopman model to predict second motion state data of the robot at a next moment, wherein the values ​​of various parameters in the Koopman model are determined based on the operating results of the robot in multiple different operating environments, including non-simulation operating environments; the various parameters include target parameters, and the importance of each target parameter in each operating environment is greater than a preset level; inputting the first motion state data, the second motion state data, and the control target into the control model to obtain a control instruction for the robot at the next moment; and controlling the robot to execute the control instruction.

[0132] In addition, the logical instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the existing technology, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program code.

[0133] On the other hand, the present application also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the robot control method based on the Koopman model provided by the above methods, the method including: obtaining the first motion state data and control target of the robot at the current moment; inputting the first motion state data into the Koopman model to predict the second motion state data of the robot at the next moment, the value of each parameter in the Koopman model is determined based on the operating results of the robot in multiple different operating environments, the operating environment including a non-simulation operating environment, the various parameters include target parameters, and the importance of each target parameter in each of the operating environments is greater than a preset level; inputting the first motion state data, the second motion state data and the control target into the control model to obtain the control instruction of the robot at the next moment; controlling the robot to execute the control instruction.

[0134] On the other hand, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the robot control method based on the Koopman model provided by the above-mentioned methods, the method comprising: obtaining the first motion state data and control target of the robot at the current moment; inputting the first motion state data into the Koopman model to predict the second motion state data of the robot at the next moment, the values ​​of each parameter in the Koopman model being determined based on the operating results of the robot in a plurality of different operating environments, the operating environment including a non-simulation operating environment, the various parameters including target parameters, the importance of each target parameter in each of the operating environments being greater than a preset degree; inputting the first motion state data, the second motion state data and the control target into the control model to obtain the control instruction of the robot at the next moment; and controlling the robot to execute the control instruction.

[0135] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0136] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0137] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A robot control method based on the Koopman model, characterized in that: A controller applied to a robot, wherein a Koopman model and a control model are deployed in the controller, and the method includes: Obtaining first motion state data and control target of the robot at a current moment; Inputting the first motion state data into the Koopman model to predict second motion state data of the robot at a next moment, wherein values ​​of various parameters in the Koopman model are determined based on operating results of the robot in a plurality of different operating environments, the operating environments including non-simulation operating environments, the various parameters including target parameters, and importance of each target parameter in each of the operating environments being greater than a preset value; Inputting the first motion state data, the second motion state data, and the control target into the control model to obtain a control instruction for the robot at the next moment; Control the robot to execute the control instruction.

2. The robot control method based on the Koopman model according to claim 1, characterized in that: The Koopman model includes an encoding module and an operator module. The first motion state data is input into the Koopman model to predict the second motion state data of the robot at the next moment, including: Inputting the first motion state data into the encoding module to obtain a state representation of the first motion state data in a target feature space, wherein the dimension of the target feature space is higher than the dimension of an original state space in which the first motion state data is located; The state representation is input into the operator module to obtain the second motion state data. The operator module is used to perform a linear transformation on the state representation through a Koopman matrix and obtain the second motion state data after the linear transformation.

3. The robot control method based on the Koopman model according to claim 1, characterized in that: The step of inputting the first motion state data into the target Koopman model to predict the second motion state data of the robot at the next moment includes: Inputting the first motion state data into a target Koopman model to predict the second motion state data of the robot at the next moment; The target Koopman model is obtained by adjusting the values ​​of target parameters in the Koopman model according to the operation results of the robot in the current working environment, and the adjusted Koopman model is the target Koopman model.

4. The robot control method based on the Koopman model according to any one of claims 1 to 3, characterized in that: The target parameters in the Koopman model are determined by the following steps: Obtaining the operation results of the robot in each of the operation environments; According to each of the operation results, a Koopman operator model corresponding to each of the operation environments is trained, and the network structure of each of the Koopman operator models is the same; Determining the importance score of each parameter in each of the Koopman operator models through an elastic weight solidification algorithm; According to the importance score, a target parameter in the Koopman model is determined.

5. The robot control method based on the Koopman model according to claim 4, characterized in that: Determining the importance score of each parameter in each Koopman operator model by the elastic weight solidification algorithm includes: Determining the Fisher information matrix corresponding to each of the Koopman operator models; The importance score of each parameter in each Koopman operator model is determined according to the diagonal elements in the Fisher information matrix.

6. The robot control method based on the Koopman model according to claim 4, characterized in that: The Koopman model has the same network structure as each of the Koopman operator models, and determining the target parameters in the Koopman model according to the importance score includes: For each parameter in the Koopman model, determining a comprehensive importance score according to the importance score of the corresponding parameter in each Koopman operator model; The parameter whose comprehensive importance score in the Koopman model is greater than the importance score threshold is determined as the target parameter.

7. The robot control method based on the Koopman model according to claim 6, characterized in that: The values ​​of the parameters in the Koopman model are determined based on the average values ​​of the corresponding parameters in the Koopman operator models, or based on the values ​​of the corresponding parameters in the Koopman operator model corresponding to an operating environment similar to the current working environment of the robot.

8. The robot control method based on the Koopman model according to claim 4, characterized in that: The control model is obtained by the following steps: Constructing a digital model of the robot and a virtual working environment in a simulator; determining a training task for the robot based on the digital model and the virtual working environment; An initial control model of the robot is obtained, and the initial control model is trained according to the training task until the initial control model meets a preset condition, and the initial control model that meets the preset condition is the control model.

9. A robot control device based on the Koopman model, characterized in that: A controller used in a robot, wherein a Koopman model and a control model are deployed in the controller, and the device comprises: A first acquisition module is used to obtain the first motion state data and control target of the robot at a current moment; a prediction module, configured to input the first motion state data into the Koopman model to predict second motion state data of the robot at a next moment, wherein values ​​of parameters in the Koopman model are determined based on operating results of the robot in a plurality of different operating environments, the operating environments including non-simulation operating environments, the parameters including target parameters, and the importance of each target parameter in each of the operating environments being greater than a preset degree; a second acquisition module, configured to input the first motion state data, the second motion state data, and the control target into the control model to obtain a control instruction for the robot at a next moment; A control module is used to control the robot to execute the control instruction.

10. A robot comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the robot control method based on the Koopman model according to any one of claims 1 to 8 is implemented.

Images