Intelligent agent control method, controller, intelligent agent and storage medium

Abstract

The embodiment of the invention discloses an intelligent agent control method, a controller, an intelligent agent and a storage medium. The method comprises the following steps: inputting a current system state and a current control input quantity into a predictive control model to obtain an action description sequence of multiple actions implemented by an intelligent agent in future time, obtaining an actual contact force of the intelligent agent in contact with a target object, generating a contact force cost item based on a difference value between the contact force sequence and the actual contact force, and executing optimization operation based on the contact force cost item to obtain a target control input sequence. According to the embodiment of the invention, the actual contact force of the intelligent agent in contact with the target object is used as feedback, the actual contact force and the contact force sequence predicted by the predictive control model are subjected to cost expression, and the difference between the contact force sequence predicted by the predictive control model and the actual contact force is reflected through the contact force cost item. And finally, the contact force cost item is optimized, so that the obtained target control input sequence can make up for the defects of the contact force sequence predicted by the prediction control model.

Description

Technical Field

[0001] This application relates to the field of intelligent agent technology, and in particular to a control method, controller, intelligent agent, and storage medium for an intelligent agent. Background Technology

[0002] When an intelligent agent performs a task, it needs to come into contact with objects; for example, it needs to grasp an object. To ensure that the agent can smoothly and effectively contact the object, related technologies use a non-complementarity model to calculate the contact force between the agent and the object. However, the contact force output by the non-complementarity model still has unavoidable deviations in the real world. For example, the actual stiffness and coefficient of friction of the object's surface are unknown to the agent. This can lead to a significant difference between the contact force output by the non-complementarity model and the actual contact force, which can easily cause the agent to fail in its task execution. Summary of the Invention

[0003] One objective of this application is to provide a control method, controller, intelligent agent, and storage medium for an intelligent agent, thereby improving the situation where the intelligent agent is prone to fail to perform tasks due to the deviation between the predicted contact force and the actual contact force in related technologies.

[0004] In a first aspect, embodiments of this application provide a control method for an intelligent agent, applied to an intelligent agent, comprising: acquiring the current system state, current control input, and a preset predictive control model of the intelligent agent, wherein the predictive control model is a control model relating to the system state, control input, and contact force of the intelligent agent, and wherein the predictive control model is differentiable with respect to the contact force; inputting the current system state and the current control input into the predictive control model to obtain an action description sequence in which the intelligent agent will perform multiple actions in the future, the action description sequence including a contact force sequence; acquiring the actual contact force of the intelligent agent contacting a target object; generating a contact force cost term based on the difference between the contact force sequence and the actual contact force; and performing an optimization operation based on the contact force cost term to obtain a target control input sequence in which the intelligent agent will perform multiple actions in the future.

[0005] Optionally, the contact force sequence includes multiple predicted contact forces, and the step of generating a contact force cost term based on the difference between the contact force sequence and the actual contact force includes: obtaining a first difference between each predicted contact force and the actual contact force; determining a quadratic weighted norm corresponding to the predicted contact force based on the first difference; and generating a contact force cost term based on the quadratic weighted norm of all predicted contact forces.

[0006] Optionally, the step of performing optimization operations based on the contact force cost term to obtain the target control input sequence for the agent to perform multiple actions in the future time includes: obtaining a constraint cost term, which is used to constrain the actions performed by the agent in the future time; and performing optimization operations based on the constraint cost term and the contact force cost term to obtain the target control input sequence for the agent to perform multiple actions in the future time.

[0007] Optionally, the action description sequence further includes a system state sequence, the constraint cost term includes a task cost term and a control penalty term, and the process of obtaining the constraint cost term includes: obtaining a preset reference state sequence; generating a task cost term based on the system state sequence and the reference state sequence; constructing a control input sequence to be determined; and generating a control penalty term based on the control input sequence to be determined.

[0008] Optionally, the system state sequence includes multiple predicted system states, the reference state sequence includes multiple reference system states, and one predicted system state corresponds to one reference system state. The step of generating a task cost term based on the system state sequence and the reference state sequence includes: obtaining a second difference between the predicted system state and the corresponding reference system state; determining a quadratic weighted norm corresponding to the predicted system state based on the second difference; and generating a task cost term based on the quadratic weighted norms corresponding to all predicted system states.

[0009] Optionally, the control input sequence to be determined includes multiple control input quantities to be determined, and the step of generating a control quantity penalty term based on the control input sequence to be determined includes: generating a quadratic weighted norm for the control input quantities to be determined; summing the quadratic weighted norms of all control input quantities to obtain a first total weighted norm; calculating a third difference between two adjacent control input quantities; determining the quadratic weighted norm of the control input quantities to be determined based on the third difference; summing the quadratic weighted norms corresponding to all control input quantities to be determined to obtain a second total weighted norm; and generating a control quantity penalty term based on the first total weighted norm and the second total weighted norm.

[0010] Optionally, the constraint cost term includes a task cost term and a control penalty term. The step of performing an optimization operation based on the constraint cost term and the contact force cost term to obtain the target control input sequence for the agent to perform multiple actions in the future includes: multiplying the control penalty term by a preset first weighting coefficient to obtain a weighted control penalty term; multiplying the contact force cost term by a preset second weighting coefficient to obtain a weighted contact force cost term; adding the task cost term, the weighted control penalty term, and the weighted contact force cost term to obtain a composite cost; and performing an optimization operation on the composite cost based on a preset optimizer to obtain the target control input sequence for the agent to perform multiple actions in the future.

[0011] In a second aspect, embodiments of this application provide a controller, including a memory and a processor, wherein the memory is connected to the processor, and the processor is configured to execute one or more computer programs stored in the memory, wherein when the processor executes the one or more computer programs, the controller enables the controller to implement the aforementioned control method for an intelligent agent.

[0012] In a third aspect, embodiments of this application provide an intelligent agent, including the controller described above.

[0013] In a fourth aspect, embodiments of this application provide a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the aforementioned control method for an intelligent agent.

[0014] The embodiments of this application can achieve the following technical effects: The embodiments of this application use the actual contact force of the intelligent agent contacting the target object as feedback, and express the actual contact force and the contact force sequence predicted by the predictive control model in a cost-based manner. The contact force cost term reflects the difference between the contact force sequence predicted by the predictive control model and the actual contact force. Finally, the contact force cost term is optimized so that the obtained target control input sequence can make up for the deficiencies of the contact force sequence predicted by the predictive control model, ensuring that the intelligent agent can reliably, accurately and effectively contact the target object, which is conducive to improving the success rate and reliability of task execution. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1This is a schematic diagram of the circuit structure of an intelligent agent provided in an embodiment of this application; Figure 2 A schematic diagram illustrating the architecture for communication between an agent and a server, as provided in an embodiment of this application. Figure 3 A flowchart illustrating a control method for an intelligent agent provided in an embodiment of this application; Figure 4 This is a schematic diagram illustrating an application scenario of an intelligent agent grasping a bottle, as provided in an embodiment of this application. Figure 5 This is a schematic diagram of the structure of a control device for an intelligent agent provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of a controller provided in an embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0018] It should be noted that, unless there is a conflict, the various features in the embodiments of this application can be combined with each other, all of which are within the protection scope of this application. Furthermore, although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than the module division in the device or the order in the flowchart. Moreover, the terms "first," "second," and "third" used in this application do not limit the data or execution order, but only distinguish identical or similar items with essentially the same function and effect.

[0019] The core challenge of Model Predictive Control (MPC) in robot contact tasks is the accuracy and computational efficiency of the contact dynamics model. Traditional contact dynamics models are often non-smooth and discontinuous, making it difficult to support gradient-based fast optimization algorithms.

[0020] To address this issue, academia and industry have designed differentiable contact models, such as non-complementarity models. Differentiable contact models approximate the contact process through smooth functions, making the entire dynamic system differentiable with respect to state and control inputs. This allows for the use of efficient gradient optimization methods to plan complex contact behaviors of robots, thus greatly promoting the application of MPC models in contact tasks.

[0021] However, even with advanced non-complementary models, unavoidable discrepancies still exist between the MPC model and the real world. For example, unknown parameters such as the actual surface stiffness, coefficient of friction, and precise location of an object can cause differences between the contact forces predicted by the MPC model and the actual contact forces. These discrepancies can reduce the robot's control accuracy and even lead to task failure.

[0022] The inventors also discovered that related technologies typically involve equipping robots with tactile sensors to acquire the actual contact force between the robot and an object. Based on this feedback, the robot adjusts its output contact force, ensuring reliable and safe contact. This solves the technical problem of discrepancies between the contact force predicted by the MPC model and the actual contact force. For example, the robot obtains the predicted contact force through an MPC model, collects the actual contact force through tactile sensors, subtracts the predicted contact force from the actual contact force to obtain the deviation, and inputs this deviation into a compensator supporting a PID algorithm. The output of the compensator adjusts the robot's actual output contact force, ensuring that the actual output contact force matches the actual contact force, thus guaranteeing reliable and safe contact with the object.

[0023] This approach, known as "pre-optimization, post-compensation," essentially treats "MPC model deviations" as "optimized interference" rather than "variables to consider during optimization." This results in the MPC model outputting predicted contact forces based on "inaccurate model parameters" during the solution process, with subsequent compensation correcting the discrepancy between the predicted and actual contact forces. This approach fails to meet the requirements of high-precision and high-real-time operation scenarios. Furthermore, when the robot moves at very high speeds, deviations accumulate, making it impossible for the compensator to correct these accumulated deviations in real time. Consequently, the robot's task execution accuracy is low, or it is prone to failure.

[0024] The following embodiments of this application provide an intelligent agent. Please refer to... Figure 1 The intelligent agent 100 includes a visual sensing module 11, a lidar module 12, a force sensor module 13, a joint sensor module 14, a controller 15, a robotic arm 16, and a communication module 17.

[0025] The visual sensing module 11 is used to acquire environmental images of the target space where the intelligent agent 100 is located. The target space can be a workplace, a home, or a public place, etc., and various objects can be placed in the target space. The intelligent agent 100 completes relevant tasks within the target space. The visual sensing module 11 includes an RGB-D camera or an industrial 3D structured light camera, etc.

[0026] The lidar module 12 is used to acquire point clouds of the target space where the intelligent agent 100 is located. The lidar module 12 includes 16-line lidar or 32-line lidar, etc.

[0027] The force sensor module 13 is used to collect the contact force of the robotic arm 16 when it contacts an object. For example, if the robotic arm 16 needs to grasp a power strip, the force sensor module 13 can detect the contact force of the robotic arm 16 grasping the power strip.

[0028] The joint sensor module 14 is used to detect the joint angles and angular velocities of each joint in the robotic arm 16. The joint sensor module 14 can be a joint encoder.

[0029] The controller 15 is communicatively connected to the vision sensing module 11, the lidar module 12, the force sensor module 13, the joint sensor module 14, and the robotic arm 16. As the control core of the intelligent agent 100, the controller 15 can control the intelligent agent 100 to perform various contact tasks, including assembly tasks, grasping tasks, or polishing tasks.

[0030] The robotic arm 16 includes multiple links, multiple joints, actuators, and an end effector. The links are connected to the joints, the actuators are connected to the joints, and the end effector is mounted on the links. The actuators are used to drive the joints to move the links based on control commands sent by the controller 15, and the end effector is used to grasp objects.

[0031] The communication module 17 is electrically connected to the controller 15 and is used for interactive communication with the server. The communication module 17 includes a WIFI communication module, an Ethernet communication module, a 6G communication module, a 5G communication module, a 4G communication module, a 3G communication module, etc.

[0032] Please see Figure 2 The intelligent agent 100 provided in this embodiment can also communicate and interact with the cloud server 200. The cloud server 200 can send control commands to the intelligent agent 100, and the intelligent agent 100 can execute contact tasks according to the control commands. At the same time, the intelligent agent 100 can also send various types of data generated during the execution of contact tasks to the cloud server 200 for storage.

[0033] The following embodiments of this application provide a control method for an intelligent agent, applied to an intelligent agent. It is understood that the intelligent agent here can be the intelligent agent with a robot structure provided in the above embodiments, or it can be a robotic arm, a dexterous hand, or an end effector, etc.

[0034] Please see Figure 3 The embodiment of this application implements a control method for an intelligent agent through steps S31 to S35, as detailed below: Step S31: Obtain the current system state, current control input, and preset predictive control model of the intelligent agent.

[0035] The current system state refers to the system state of the agent at the current time. The system state describes the agent's working state. The system state includes the agent's pose, joint angles, joint velocities, etc.

[0036] The current control input is the control input for the current control agent to perform the contact task. The control input includes joint torque, joint velocity, joint angular velocity, etc.

[0037] The predictive control model is a control model concerning the system state, control input, and contact force of the agent, wherein the predictive control model is differentiable with respect to the contact force. The expression for the predictive control model is: .in, Let t be the system state at time t. Let be the control input at time t. Let be the contact force function at time t. The contact force function represents the object properties of the object the agent comes into contact with, including object position, stiffness, and coefficient of friction. The contact force function is a continuous function and is differentiable with respect to the contact force.

[0038] In some embodiments, the expression for the contact force function is: Where k is the elastic coefficient, For smoothing coefficients, For normal deformation, where, normal deformation It is related to the system state (such as pose) of the agent. As can be seen from the expression of the contact force function, the contact force function is a continuous function, and the contact force can be differentiated.

[0039] In other embodiments, the expression for the contact force function is: .in, This is the damping coefficient. In this embodiment, a damping adjustment force related to normal deformation is added to the original contact force function, making the contact between the agent and the object buffered, ensuring safer and more reliable contact between the agent and the object.

[0040] Step S32: Input the current system state and the current control input into the predictive control model to obtain the action description sequence of the agent to perform multiple actions in the future.

[0041] The action description sequence describes the contact forces and / or system states when the agent performs each action in the future. In some embodiments, the action description sequence includes a contact force sequence, which is a set of contact forces when the agent performs multiple actions in the future. The contact force sequence includes multiple predicted contact forces, which are the contact forces exerted by the agent when it contacts an object, given by the predictive control model, and each predicted contact force corresponds to a future time point.

[0042] In some embodiments, the action description sequence further includes a system state sequence, which is a set of system states in which the agent performs multiple actions in the future. The system state sequence includes multiple reference system states, which provide the predictive control model with the system state when the agent comes into contact with an object, and each reference system state corresponds to a future point in time.

[0043] This application embodiment inputs the current system state and the current control input quantity into a predictive control model, and uses the predictive control model to perform forward simulation, thereby obtaining a reference system state and predicted contact force for the agent to perform N actions in the future. For example, this application embodiment inputs the current system state... With current control input Input predictive control model The following relationships exist: Formula 1 As shown in Equation 1, the predictive control model can output a contact force sequence. and system state sequence .

[0044] Step S33: Obtain the actual contact force between the intelligent agent and the target object.

[0045] The target object is the object that the intelligent agent is required to touch in the contact task. For example, the target object is a bottle, an egg, or a box. The contact task requires the intelligent agent to hold the bottle, grab the egg, or carry the box. It is understood that the "contact" mentioned in the embodiments of this application includes not only the narrow understanding of the intelligent agent's surface touching the surface of the object, but also the broad understanding of the intelligent agent needing to touch the object due to actions such as holding the object, grabbing the object, or lifting the object.

[0046] The actual contact force is the contact force when the agent comes into contact with the target object. The agent is equipped with a force sensor module. When the agent comes into contact with the target object, it makes contact with the target object through the force sensor module. Therefore, the force sensor module generates the actual contact force, and the agent obtains the actual contact force through the force sensor module.

[0047] Step S34: Generate a contact force cost term based on the difference between the contact force sequence and the actual contact force.

[0048] The contact force cost term is used to instruct the predictive control model to internally adjust the predicted contact force to optimize the deviation between the output predicted contact force and the actual contact force. The contact force sequence includes multiple predicted contact forces, and embodiments of this application can generate the contact force cost term based on the predicted contact force and the actual contact force.

[0049] In some embodiments, generating a contact force cost term based on the difference between the contact force sequence and the actual contact force includes the following steps: obtaining a first difference between each predicted contact force and the actual contact force, determining a quadratic weighted norm corresponding to the predicted contact force based on the first difference, and generating a contact force cost term based on the quadratic weighted norm of all predicted contact forces.

[0050] In this embodiment of the application, the first difference between each predicted contact force and the actual contact force is calculated according to Formula 2, as shown below: Formula 2 in, The predicted contact force at time t+i. For actual contact force, This is the first difference corresponding to the predicted contact force at time t+i.

[0051] In this embodiment, the second-order weighted norm corresponding to the predicted contact force is calculated according to Equation 3, as shown below: Formula 3 in, To predict contact force The corresponding quadratic weighted norm, Let be the first non-negative definite weight matrix. It is a double-weighted square.

[0052] In this embodiment of the application, the contact force cost term is obtained according to Equation 4, as shown below: Formula 4 in, This is the contact force cost term.

[0053] In this embodiment, the predicted contact force is subtracted from the actual contact force, and the first difference of the subtraction result is used to construct a contact force cost term. This helps to make the control input obtained in subsequent steps smoother, so as to output the predicted contact force in a smooth manner. The predicted contact force can change smoothly so as to be consistent with the actual contact force, avoiding drastic fluctuations when the intelligent agent contacts the target object.

[0054] Step S35: Perform optimization operation based on the contact force cost term to obtain the target control input sequence for the agent to perform multiple actions in the future.

[0055] In this embodiment, an optimizer can be used to optimize the contact force cost term to obtain the target control input sequence for the agent to perform multiple actions in the future. The optimizer can be an optimizer based on SQP (Sequential Quadratic Programming) or an optimizer based on Multi-Level SQP, etc.

[0056] In this embodiment, the actual contact force of the agent contacting the target object is used as feedback. The actual contact force and the contact force sequence predicted by the predictive control model are expressed in a cost-based manner. The contact force cost term reflects the difference between the contact force sequence predicted by the predictive control model and the actual contact force. Finally, the contact force cost term is optimized so that the obtained target control input sequence can make up for the deficiencies in the contact force sequence predicted by the predictive control model, ensuring that the agent can reliably, accurately and effectively contact the target object, which is beneficial to improving the success rate and reliability of task execution.

[0057] It is also understandable that the contact force near the singularity point is prone to drastic fluctuations in traditional methods. However, in the embodiments of this application, the predictive control model is differentiable with respect to the contact force. The optimizer can calculate the gradient of the contact force cost term through the chain rule and calculate the target control input sequence for the agent to perform multiple actions in the future based on the gradient of the contact force cost term. Therefore, the contact force output by the agent changes smoothly and will not change drastically, which is beneficial to improving the execution stability and reliability of the contact task.

[0058] When an agent performs a contact task, it is susceptible to interference. To ensure the agent can reliably and safely perform the contact task, this embodiment optimizes the constraint cost term and the contact force cost term together when obtaining the target control input sequence. Specifically, this embodiment performs optimization operations based on the contact force cost term through steps S351 and S352 to obtain the target control input sequence for the agent to perform multiple actions in the future, as shown below: Step S351: Obtain the constraint cost term.

[0059] The constraint cost term is used to constrain the actions the agent will take in the future. The constraint cost term includes a task cost term and a control penalty term. The task cost term corrects deviations that occur when the agent performs the contact task, ensuring that the agent can complete the contact task safely and reliably. The control penalty term constrains the agent's control inputs to prevent the agent from taking drastic actions.

[0060] Obtaining the constraint cost term includes the following steps: obtaining a preset reference state sequence, generating a task cost term based on the system state sequence and the reference state sequence, constructing a control input sequence to be determined, and generating a control penalty term based on the control input sequence to be determined.

[0061] The reference state sequence is a pre-planned system state sequence for each moment in a contact task. For example, if a contact task requires the agent to grab a bottle, the agent pre-plans the system states for each moment in the reference state sequence. Subsequently, the agent performs actions according to each system state in the reference state sequence to complete the contact task.

[0062] The timestamps of the reference state sequence are aligned with the timestamps of the system state sequence, where the expression for the reference state sequence is: The embodiments of this application can invoke a reference state sequence locally from the intelligent agent.

[0063] The system state sequence includes multiple predicted system states, and the reference state sequence includes multiple reference system states. One predicted system state corresponds to one reference system state. The task cost term is generated based on the system state sequence and the reference state sequence, which includes the following steps: obtaining the second difference between the predicted system state and the corresponding reference system state; determining the quadratic weighted norm corresponding to the predicted system state based on the second difference; and generating the task cost term based on the quadratic weighted norm corresponding to all predicted system states.

[0064] In this embodiment of the application, the second difference between the predicted system state and the corresponding reference system state is calculated according to Equation 5, as shown below: Formula 5 in, For the predicted system state at time t+i, Let be the reference system state at time t+i. This is the second difference corresponding to the predicted system state at time t+i.

[0065] In this embodiment of the application, the quadratic weighted norm corresponding to the system state is calculated and predicted according to Equation 6, as shown below: Formula Six in, To predict the system state The corresponding quadratic weighted norm, This is the second non-negative definite weight matrix.

[0066] This embodiment of the application obtains the task cost term according to Formula 7, as shown below: Formula 7 in, This is the cost of the task.

[0067] In this embodiment, the predicted system state is subtracted from the reference system state, and the second difference of the subtraction result is used to construct the task cost term, which helps the agent to reliably complete the contact task.

[0068] The desired control input sequence includes multiple desired control input quantities, which are the control input quantities that need to be obtained. Generating a control penalty term based on the desired control input sequence includes the following steps: Based on the desired control input quantities, generate a quadratic weighted norm for each desired control input quantity; sum the quadratic weighted norms of all control input quantities to obtain a first total weighted norm; calculate the third difference between two adjacent control input quantities; determine the quadratic weighted norm of each desired control input quantity based on the third difference; sum the quadratic weighted norms corresponding to all desired control input quantities to obtain a second total weighted norm; and generate a control penalty term based on the first and second total weighted norms.

[0069] This application embodiment generates a second-order weighted norm with respect to the control input quantity according to formula eight, as shown below: Formula 8 in, Let be the control input quantity to be determined at time t+i. This is the third non-negative definite weight matrix. Let be the quadratic weighted norm of the control input variable to be determined at time t+i.

[0070] The embodiment of this application obtains the first total weighted norm according to Equation 9, as shown below: Formula Nine in, The first total weighted norm, This is the third non-negative definite weight matrix.

[0071] In this embodiment of the application, the third difference between two adjacent control input quantities is obtained according to formula ten, as shown below: Formula 10 in, This is the third difference.

[0072] In this embodiment, the quadratic weighted norm of the control input to be determined is generated according to Equation 11, as shown below: Formula 11 in, This is the fourth non-negative definite weight matrix. Let be the quadratic weighted norm of the control input variable to be determined at time t+i.

[0073] In this embodiment, the second total weighted norm is obtained according to formula 12, as shown below: Formula Twelve in, This is the second total weighted norm.

[0074] This application embodiment obtains the control quantity penalty term according to Formula Thirteen, as shown below: Formula Thirteen in, This is a penalty item for controlling quantity.

[0075] In this embodiment, a first total weighted norm is generated using the control input quantity to be determined itself, and a second total weighted norm is generated using the third difference between two adjacent control input quantities. Then, a control quantity penalty term is generated based on the first and second total weighted norms. The control quantity penalty term can be incorporated into the control input quantity to be determined itself to avoid the control input quantity to be obtained later from going out of control, and can also be incorporated into the third difference between two adjacent control input quantities to ensure that the actions implemented by the agent are relatively smooth.

[0076] Step S352: Perform optimization operations based on constraint cost terms and contact force cost terms to obtain the target control input sequence for the agent to perform multiple actions in the future.

[0077] This embodiment integrates the task cost term, control penalty term, and contact force cost term, and inputs the integrated result into the predictive control model to obtain the target control input sequence for the agent to perform multiple actions in the future. Specifically, this embodiment performs optimization operations based on the constraint cost term and contact force cost term through steps S3521 to S3524 to obtain the target control input sequence for the agent to perform multiple actions in the future, as shown below: Step S3521: Multiply the control quantity penalty term by a preset first weighting coefficient to obtain the weighted control quantity penalty term.

[0078] In this embodiment of the application, the weighted control quantity penalty term is obtained according to formula fourteen, as shown below: Formula Fourteen in, For weighted control quantity penalty items, This is the first weighting coefficient.

[0079] Step S3522: Multiply the contact force cost term by a preset second weighting coefficient to obtain the weighted contact force cost term.

[0080] In this embodiment of the application, the weighted contact force cost term is obtained according to Formula 15, as shown below: Formula 15 in, For the weighted contact force cost term, This is the second weighting coefficient.

[0081] Step S3523: Add the task cost, weighted control penalty, and weighted contact force cost together to obtain the composite cost.

[0082] The composite cost is obtained according to Formula Sixteen in this embodiment, as shown below: Formula Sixteen in, For compound costs.

[0083] Step S3524: Based on a preset optimizer, optimize the composite cost to obtain the target control input sequence for the agent to perform multiple actions in the future.

[0084] In this embodiment, an optimizer can be used to perform optimization operations on the composite cost to obtain the target control input sequence for the agent to perform multiple actions in the future. .

[0085] This application embodiment integrates the task cost term, the weighted control quantity penalty term, and the weighted contact force cost term. After optimizing the resulting composite cost, the output target control input sequence can ensure that the agent can reliably and accurately execute the contact task, guarantee the contact task can be performed, ensure that the action during the execution of the contact task is not too volatile, smoothly execute the contact task, and further ensure that the output predicted contact force is consistent with the actual contact force, so as to reliably and safely contact the target object.

[0086] To gain a detailed understanding of the embodiments of this application, the embodiments of this application are combined with... Figure 4 This is explained in detail below: like Figure 4 As shown, robot 40 grasps bottle 41 using a robotic arm. A force sensor at the end of the robotic arm collects the actual contact force between the end of the robotic arm and bottle 41. Robot 40 generates a contact force cost term based on the actual contact force and the predicted contact force. It adds the task cost term, the weighted control penalty term, and the weighted contact force cost term to obtain a composite cost. The composite cost is then optimized using an optimizer to obtain the target control input sequence for the agent to perform multiple actions in the future. In this embodiment, before outputting the target control sequence, the deviation between the predicted contact force and the actual contact force is actively corrected to ensure that the contact force generated based on the target control input sequence is consistent with the actual contact force. This allows the agent to contact the target object in a manner that is nearly or completely matched to the actual contact force, which is beneficial to improving the success rate, accuracy, and safety of the contact task.

[0087] In summary, the embodiments of this application have at least the following technical effects: 1) In this embodiment of the application, multiple predicted contact forces of the agent in the future are predicted by a predictive control model. Then, based on the feedback of the actual contact force, a contact force cost term is generated by the first difference between the predicted contact force and the actual contact force. The contact force cost term is optimized, thereby realizing the active correction of the deviation between the predicted contact force and the actual contact force before the target control input sequence is output. This ensures that the contact force generated according to the target control input sequence is consistent with the actual contact force, so that the agent contacts the target object in a way that is close to or completely matches the actual contact force. This is beneficial to improving the success rate, accuracy and safety of the contact task.

[0088] 2) Differentiable predictive models are used to avoid fluctuations in the control input of the agent at singular points, ensuring that the agent can perform contact tasks with high precision, high stability and high real-time performance.

[0089] 3) Within the framework of the predictive control model, a rapid closed loop of "prediction-perception-decision-execution" is achieved, taking into account both the foresight and responsiveness of the agent, making the agent more reliable and efficient.

[0090] It should be noted that in the above embodiments, there is no necessarily a certain order between the steps. Those skilled in the art can understand from the description of the embodiments of this application that the above steps may have different execution orders in different embodiments, that is, they may be executed in parallel or in turn, etc.

[0091] As another aspect of the embodiments of this application, this application provides a control device for an intelligent agent. The control device for the intelligent agent can be a software module, which includes several instructions stored in a memory. A processor can access the memory, invoke the instructions, and execute them to complete the intelligent agent control methods described in the various embodiments above.

[0092] In some embodiments, the control device for the intelligent agent can also be constructed from hardware devices. For example, the control device for the intelligent agent can be constructed from one or more chips, which can work in coordination to complete the control method for the intelligent agent described in the various embodiments above. As another example, the control device for the intelligent agent can also be constructed from various logic devices, such as general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), microcontrollers, ARM (Acorn RISC Machine) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination of these components.

[0093] Please see Figure 5 The control device 500 for the intelligent agent includes a parameter acquisition module 51, an action prediction module 52, a contact force acquisition module 53, a contact cost module 54, and an optimization module 55.

[0094] The parameter acquisition module 51 is used to acquire the current system state, current control input, and preset predictive control model of the agent. The predictive control model is a control model for the system state, control input, and contact force of the agent. The predictive control model is differentiable with respect to the contact force. The action prediction module 52 is used to input the current system state and current control input into the predictive control model to obtain an action description sequence of multiple actions to be performed by the agent in the future. The action description sequence includes a contact force sequence. The contact force acquisition module 53 is used to acquire the actual contact force of the agent when it contacts the target object. The contact cost module 54 is used to generate a contact force cost term based on the difference between the contact force sequence and the actual contact force. The optimization module 55 is used to perform optimization operations based on the contact force cost term to obtain the target control input sequence of multiple actions to be performed by the agent in the future.

[0095] This application embodiment predicts multiple predicted contact forces of the agent in the future time using a predictive control model. Then, based on the feedback of the actual contact force, a contact force cost term is generated using the first difference between the predicted contact force and the actual contact force. The contact force cost term is then optimized, thereby enabling the active correction of the deviation between the predicted contact force and the actual contact force before the target control input sequence is output. This ensures that the contact force generated based on the target control input sequence is consistent with the actual contact force, allowing the agent to contact the target object in a way that is close to or completely matches the actual contact force. This is beneficial for improving the success rate, accuracy, and safety of the contact task.

[0096] In some embodiments, the contact force sequence includes multiple predicted contact forces, and the contact cost module 54 is specifically used to: obtain a first difference between each predicted contact force and the actual contact force, determine the second-order weighted norm corresponding to the predicted contact force based on the first difference, and generate a contact force cost term based on the second-order weighted norm of all predicted contact forces.

[0097] In some embodiments, the optimization module 55 is specifically used to: obtain a constraint cost term, which is used to constrain the actions to be performed by the agent in the future, and perform an optimization operation based on the constraint cost term and the contact force cost term to obtain a target control input sequence for the agent to perform multiple actions in the future.

[0098] In some embodiments, the action description sequence further includes a system state sequence, the constraint cost term includes a task cost term and a control penalty term, and the optimization module 55 is further specifically used to: obtain a preset reference state sequence, generate a task cost term based on the system state sequence and the reference state sequence, construct a control input sequence to be determined, and generate a control penalty term based on the control input sequence to be determined.

[0099] In some embodiments, the system state sequence includes multiple predicted system states, the reference state sequence includes multiple reference system states, and one predicted system state corresponds to one reference system state. The optimization module 55 is further specifically used to: obtain a second difference between the predicted system state and the corresponding reference system state, determine the quadratic weighted norm corresponding to the predicted system state based on the second difference, and generate a task cost term based on the quadratic weighted norm corresponding to all predicted system states.

[0100] In some embodiments, the control input sequence to be determined includes multiple control input quantities to be determined, and the optimization module 55 is further specifically used for: generating a quadratic weighted norm of the control input quantities to be determined based on the control input quantities to be determined; summing the quadratic weighted norms of all control input quantities to obtain a first total weighted norm; calculating a third difference between two adjacent control input quantities; determining the quadratic weighted norm of the control input quantities to be determined based on the third difference; summing the quadratic weighted norms corresponding to all control input quantities to be determined to obtain a second total weighted norm; and generating a control quantity penalty term based on the first total weighted norm and the second total weighted norm.

[0101] In some embodiments, the constraint cost term includes a task cost term and a control penalty term. An optimization operation is performed based on the constraint cost term and the contact force cost term to obtain a target control input sequence for the agent to perform multiple actions in the future. This includes: multiplying the control penalty term by a preset first weighting coefficient to obtain a weighted control penalty term; multiplying the contact force cost term by a preset second weighting coefficient to obtain a weighted contact force cost term; adding the task cost term, the weighted control penalty term, and the weighted contact force cost term to obtain a composite cost; and optimizing the composite cost based on a preset optimizer to obtain a target control input sequence for the agent to perform multiple actions in the future.

[0102] It should be noted that the control device for the intelligent agent described above can execute the control method for the intelligent agent provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects of the method. Technical details not described in detail in the embodiments of the control device for the intelligent agent can be found in the control method for the intelligent agent provided in the embodiments of this application.

[0103] See Figure 6 , Figure 6This is a schematic diagram of a controller provided in an embodiment of this application. The controller 15 includes one or more processors 151 and a memory 152. The memory 152 is connected to one or more processors 151, for example, via a bus.

[0104] Processor 151 is configured to support the controller in performing the corresponding functions in the methods described in the above method embodiments. The processor may be a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof. The aforementioned hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0105] Memory 152 is used to store program code, etc. Memory may include volatile memory (VM), such as random access memory (RAM); memory may also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); memory may also include combinations of the above types of memory.

[0106] The memory 152 can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the control method of the intelligent agent in the embodiments of this application. The processor executes the control method of the intelligent agent and various functional applications and data processing of the control device of the intelligent agent by running the non-volatile software programs, instructions, and modules stored in the memory, that is, it realizes the functions of the control method of the intelligent agent and the various modules or units of the control device of the intelligent agent provided in the above method embodiments.

[0107] The memory 152 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function. The data storage area may store data created based on the use of the agent's control device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the agent's control device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0108] The one or more modules are stored in the memory. When executed by the one or more processors, they execute the control method of the intelligent agent in any of the above method embodiments. For example, they execute the method steps described in the above method embodiments to realize the functions of the modules described in the above device embodiments.

[0109] This application also provides a computer-readable storage medium storing a computer program, the computer program including program instructions, which, when executed by a controller, cause the controller to perform the method described in the foregoing embodiments.

[0110] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0111] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A control method for an intelligent agent, applied to an intelligent agent, characterized in that, include: The current system state, current control input, and preset predictive control model of the agent are obtained. The predictive control model is a control model with respect to the system state, control input, and contact force of the agent, wherein the predictive control model is differentiable with respect to the contact force. The current system state and the current control input are input into the predictive control model to obtain a sequence of action descriptions of the agent performing multiple actions in the future time, the sequence of action descriptions including a contact force sequence. Obtain the actual contact force between the intelligent agent and the target object; A contact force cost term is generated based on the difference between the contact force sequence and the actual contact force; An optimization operation is performed based on the contact force cost term to obtain the target control input sequence for the agent to perform multiple actions in the future.

2. The control method according to claim 1, characterized in that, The contact force sequence includes multiple predicted contact forces, and the generation of a contact force cost term based on the difference between the contact force sequence and the actual contact force includes: Obtain a first difference between each predicted contact force and the actual contact force; Based on the first difference, determine the second-weighted norm corresponding to the predicted contact force; The contact force cost term is generated based on the quadratic weighted norm of all predicted contact forces.

3. The control method according to claim 1 or 2, characterized in that, The optimization operation based on the contact force cost term yields a target control input sequence for the agent to perform multiple actions in the future, including: Obtain a constraint cost term, which is used to constrain the actions that the agent will perform in the future. An optimization operation is performed based on the constraint cost term and the contact force cost term to obtain the target control input sequence for the agent to perform multiple actions in the future.

4. The control method according to claim 3, characterized in that, The action description sequence also includes a system state sequence, the constraint cost term includes a task cost term and a control penalty term, and the acquisition of the constraint cost term includes: Obtain the preset reference state sequence; A task cost term is generated based on the system state sequence and the reference state sequence; Construct the control input sequence to be determined; A control penalty term is generated based on the control input sequence to be determined.

5. The control method according to claim 4, characterized in that, The system state sequence includes multiple predicted system states, and the reference state sequence includes multiple reference system states. Each predicted system state corresponds to one reference system state. Generating a task cost term based on the system state sequence and the reference state sequence includes: Obtain the second difference between the predicted system state and the corresponding reference system state; Based on the second difference, determine the quadratic weighted norm corresponding to the state of the predicted system; The task cost term is generated based on the quadratic weighted norm corresponding to all predicted system states.

6. The control method according to claim 4, characterized in that, The control input sequence to be determined includes multiple control input quantities to be determined, and the generation of a control quantity penalty term based on the control input sequence to be determined includes: Based on the control input quantity to be determined, a second-order weighted norm is generated for the control input quantity to be determined; The first total weighted norm is obtained by summing the second-order weighted norms of all control inputs. Calculate the third difference between two adjacent control input values; The second-order weighted norm of the control input to be determined is determined based on the third difference. The second total weighted norm is obtained by summing the second weighted norms corresponding to all control inputs to be determined. A control penalty term is generated based on the first total weighted norm and the second total weighted norm.

7. The control method according to claim 3, characterized in that, The constraint cost term includes a task cost term and a control penalty term. The optimization operation based on the constraint cost term and the contact force cost term yields a target control input sequence for the agent to perform multiple actions in the future, including: Multiply the control quantity penalty term by a preset first weighting coefficient to obtain the weighted control quantity penalty term; Multiply the contact force cost term by a preset second weighting coefficient to obtain the weighted contact force cost term; The composite cost is obtained by adding the task cost term, the weighted control quantity penalty term, and the weighted contact force cost term. The composite cost is optimized using a preset optimizer to obtain the target control input sequence for the agent to perform multiple actions in the future.

8. A controller, characterized in that, The system includes a memory and a processor, the memory being connected to the processor, the processor being configured to execute one or more computer programs stored in the memory, and the processor, when executing the one or more computer programs, causing the controller to implement the control method for the intelligent agent as described in any one of claims 1-7.

9. An intelligent agent, characterized in that, Includes the controller as described in claim 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the control method of the intelligent agent as described in any one of claims 1-7.

Images