Method, equipment and medium for optimizing the manipulation performance of articulated objects of dexterous arm-hand robot

Abstract

The present invention discloses a method, device and medium for optimizing the operation performance of articulated objects of a dexterous arm-hand robot, belonging to the field of robot control. The method includes: step 1, constructing an environmental point cloud data set, completing the robot finger point cloud, and segmenting the point cloud; step 2, encoding the task into an LTL formula, checking the task progress with an LTL series, and encoding the LTL formula into an LTL representation with a Transformer model; step 3, establishing a contact planner, extracting point cloud features from the segmented point cloud, predicting the best contact point of the task, and optimizing the contact point prediction with the operation success rate; step 4, using an online diffusion decision model to make the intelligent body output action interact with the environment, and optimizing the decision of the diffusion decision model with sampled data; step 5, repeating steps 1 to 4 to complete the intelligent body training; step 6, using the intelligent body to control the robot to perform the articulated task. The present invention is superior to most existing methods in terms of operation ability, generalization performance, transfer reasoning and robustness.

Description

Technical Field

[0001] The invention relates to the technical field of dexterous arm-hand robot control, and in particular to a method for optimizing the operating performance of a dexterous arm-hand robot on an articulated object based on a diffusion strategy. Background Art

[0002] Manipulating articulated objects has always been an interesting and important topic in robotic learning. Although previous research has made progress in manipulating rigid bodies, challenges still exist when dealing with articulated objects, and generalizing to various types of articulated objects is particularly difficult for dexterous arm-hand robots.

[0003] Vision-based reinforcement learning is a technology that combines computer vision and reinforcement learning, which enables the agent to perceive the environment through visual sensors (such as cameras) and take actions based on visual features to maximize rewards. With the continuous development of computer vision, visual reinforcement learning algorithms based on point cloud information provide robots with rich three-dimensional geometric information, enabling them to handle objects of diverse shapes and structures. However, most existing algorithms are trying to understand the geometry of articulated objects, making it challenging for agents to determine the appropriate contact points with objects.

[0004] Affordance learning is an effective method to improve the success rate of interaction between intelligent agents and manipulated objects. The robot perceives objects in the environment with sensors (such as cameras or tactile sensors) and predicts the operation confidence map of the object through confidence learning to guide the intelligent agent to achieve effective interaction. However, generating high-quality confidence maps usually requires a large number of training interactions, which significantly affects the learning efficiency of complex manipulation tasks.

[0005] Linear temporal logic (LTL) can describe logically organized robotic tasks to guide intelligent agents to complete complex manipulation tasks. By encoding LTL as an automaton, reward machine, or LTL representation, reinforcement learning can be guided from the task level to achieve complex tasks and improve the learning efficiency of robots. However, LTL-guided motion planning itself cannot improve the generalization ability of robot manipulation skills.

[0006] The diffusion decision model (DM), as a generative model, aims to generate different samples based on a clean data distribution using a parameterized estimator. With the continuous development of robotics, DM can be designed as a diffusion policy (DP) to combine with reinforcement learning to drive the robot and the environment to achieve action interaction so as to better generalize to different tasks. However, existing DP algorithms are usually based on certain prior knowledge - using offline RL to solve problems or learning from demonstrations. Such a setting is difficult to apply to dexterous manipulation because the relevant teaching process is difficult to collect and costly.

[0007] In view of this, the present invention is proposed. Summary of the invention

[0008] The purpose of the present invention is to provide a method, device and medium for optimizing the operation performance of articulated objects of a dexterous arm-hand robot, which can improve the generalization performance of the operation of articulated objects of a dexterous arm-hand robot, improve the applicable types of articulated objects, and thus solve the above-mentioned technical problems existing in the prior art.

[0009] The objective of the present invention is achieved through the following technical solutions:

[0010] A method for optimizing the manipulation performance of an articulated object of a dexterous arm-hand robot, comprising:

[0011] Step 1: construct an environmental point cloud dataset of the environment where the dexterous arm-hand robot's articulated object manipulation task is located, complete the finger point cloud of the dexterous arm-hand robot, and segment the environmental point cloud dataset to obtain a segmented point cloud;

[0012] Step 2, encoding the articulated object manipulation task into a linear time logic formula, checking the progress of the articulated object manipulation task at each moment by using the encoded linear time logic formula through the linear time logic series, and encoding the linear time logic formula at the current moment into a linear time logic representation by using the Transformer model;

[0013] Step 3, establishing a contact planner through confidence learning, wherein the contact planner extracts point cloud features from the segmented point cloud obtained in step 1, predicts the optimal contact point of the articulated object operation task from the point cloud features, and then optimizes the prediction of the contact point using the operation success rate;

[0014] Step 4, enabling the intelligent agent controlling the dexterous arm-hand robot to sample the interaction between the online diffusion decision model output action and the environment, and optimizing the decision of the diffusion decision model according to the sampled data;

[0015] Step 5, repeating steps 1 to 4 until the training of the intelligent agent for controlling the dexterous arm-hand robot is completed;

[0016] Step 6: Use the trained intelligent agent to control the dexterous arm-hand robot to perform the task of manipulating articulated objects.

[0017] A processing device, comprising:

[0018] at least one memory for storing one or more programs;

[0019] At least one processor can execute one or more programs stored in the memory. When the one or more programs are executed by the processor, the processor can implement the method described in the present invention.

[0020] A readable storage medium stores a computer program, which can implement the method described in the present invention when the computer program is executed by a processor.

[0021] Compared with the prior art, the method, device and medium for optimizing the operation performance of articulated objects of a dexterous arm-hand robot provided by the present invention have the following beneficial effects:

[0022] The diffusion strategy is enhanced by using linear time logic representation and confidence learning, thereby improving the learning efficiency and generalization of dexterous arm-hand robot operations. Specifically, the present invention understands the task semantics through linear time logic representation, designs a contact planner to determine the optimal contact points, and then generalizes these contact points to various categories through the diffusion strategy. Based on the interaction data, the present invention further updates the decision of the diffusion strategy from the perspective of action optimization, which can well overcome the limitations of the traditional diffusion strategy. Compared with traditional skill learning methods, the present invention is superior to most existing methods in terms of operational ability, generalization performance, transfer reasoning and robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0024] Figure 1 A flow chart of a method for optimizing the articulated object manipulation performance of a dexterous arm-hand robot provided in an embodiment of the present invention.

[0025] Figure 2 A schematic diagram of the architecture of a method for optimizing the articulated object manipulation performance of a dexterous arm-hand robot provided in an embodiment of the present invention.

[0026] Figure 3 A visual schematic diagram of task environment point cloud segmentation and robot point cloud completion in the method provided in an embodiment of the present invention.

[0027] Figure 4 A visualized schematic diagram of the forward denoising process and the reverse denoising process of the diffusion decision model in the method provided in an embodiment of the present invention.

[0028] Figure 5 A schematic diagram of the architecture of the interaction and update of the diffusion decision model in the method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in combination with the specific content of the present invention; it is obvious that the described embodiments are only part of the embodiments of the present invention, not all of the embodiments, which does not constitute a limitation of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of the present invention.

[0030] First, the terms that may be used in this article are explained as follows:

[0031] The term “and / or” means that either or both of them can be realized at the same time. For example, X and / or Y means both “X” or “Y” and “X and Y”.

[0032] The terms "include", "comprises", "contains", "has" or other descriptions with similar semantics should be interpreted as non-exclusive inclusion. For example, including certain technical feature elements (such as raw materials, components, ingredients, carriers, dosage forms, materials, dimensions, parts, components, mechanisms, devices, steps, procedures, methods, reaction conditions, processing conditions, parameters, algorithms, signals, data, products or products, etc.) should be interpreted as including not only certain technical feature elements explicitly listed, but also other technical feature elements known in the art that are not explicitly listed.

[0033] The term "consisting of..." means excluding any technical feature elements not explicitly listed. If this term is used in a claim, it will make the claim closed, so that it does not contain technical feature elements other than the technical feature elements explicitly listed, except for the conventional impurities related to them. If this term only appears in a clause of a claim, it only limits the elements explicitly listed in the clause, and the elements recorded in other clauses are not excluded from the overall claim.

[0034] Unless otherwise specified or limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example: it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in this article can be understood according to specific circumstances.

[0035] When concentration, temperature, pressure, size or other parameters are expressed in the form of a numerical range, the numerical range should be understood to specifically disclose all ranges formed by the pairing of any upper limit, lower limit, and preferred value in the numerical range, regardless of whether the range is explicitly stated; for example, if a numerical range of "2 to 8" is stated, the numerical range should be interpreted as including ranges such as "2 to 7", "2 to 6", "5 to 7", "3 to 4 and 6 to 7", "3 to 5 and 7", "2 and 5 to 7", etc. Unless otherwise specified, the numerical ranges stated herein include both their end values ​​and all integers and fractions within the numerical range.

[0036] The orientation or position relationship indicated by terms such as "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", etc. are based on the orientation or position relationship shown in the drawings and are only for the convenience and simplification of description, and do not explicitly or implicitly indicate that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation of this document.

[0037] The scheme provided by the present invention is described in detail below. The contents not described in detail in the embodiments of the present invention belong to the prior art known to professionals in the field. If the specific conditions are not specified in the embodiments of the present invention, they are carried out according to the conventional conditions in the field or the conditions recommended by the manufacturer. If the manufacturers are not specified in the reagents or instruments used in the embodiments of the present invention, they are all conventional products that can be purchased commercially.

[0038] like Figure 1 As shown, an embodiment of the present invention provides a method for optimizing the operation performance of an articulated object of a dexterous arm-hand robot, comprising:

[0039] Step 1: construct an environmental point cloud dataset of the environment where the dexterous arm-hand robot's articulated object manipulation task is located, complete the finger point cloud of the dexterous arm-hand robot, and segment the environmental point cloud dataset to obtain a segmented point cloud;

[0040] Step 2, encoding the articulated object manipulation task into a linear time logic formula, checking the progress of the articulated object manipulation task at each moment by using the encoded linear time logic formula through the linear time logic series, and encoding the linear time logic formula at the current moment into a linear time logic representation by using the Transformer model;

[0041] Step 3, establishing a contact planner through confidence learning, wherein the contact planner extracts point cloud features from the segmented point cloud obtained in step 1, predicts the optimal contact point of the articulated object operation task from the point cloud features, and then optimizes the prediction of the contact point using the operation success rate;

[0042] Step 4, enabling the intelligent agent controlling the dexterous arm-hand robot to sample the interaction between the online diffusion decision model output action and the environment, and optimizing the decision of the diffusion decision model according to the sampled data;

[0043] Step 5, repeating steps 1 to 4 until the training of the intelligent agent for controlling the dexterous arm-hand robot is completed;

[0044] Step 6: Use the trained intelligent agent to control the dexterous arm-hand robot to perform the task of manipulating articulated objects.

[0045] Preferably, in step 1 of the above method, an environmental point cloud dataset of the environment where the dexterous arm-hand robot's articulated object manipulation task is located is constructed in the following manner, the finger point cloud of the dexterous arm-hand robot is completed, and the environmental point cloud dataset is segmented to obtain a segmented point cloud, including:

[0046] Controlling the intelligent agent of the dexterous arm-hand robot to randomly act in a simulation environment to construct an environmental point cloud dataset, the environmental point cloud dataset including: point clouds of functional parts of the articulated object, point clouds of other parts of the articulated object, point clouds of the dexterous hand of the dexterous arm-hand robot and point clouds of the robotic arm;

[0047] According to the robot kinematic model and joint positions, the potential posture of each finger of the dexterous hand of the dexterous arm-hand robot is calculated by forward kinematics, and the potential posture is used to sample points from the grid connected by each finger to synthesize the imaginary point cloud of the corresponding finger, and the imaginary point cloud of the corresponding finger is used to complete the corresponding finger point cloud of the dexterous hand of the dexterous arm-hand robot when the monocular camera in the environmental point cloud data set is blocked;

[0048] Pre-training PointNet by segmenting objects using the constructed environment point cloud dataset , so that the intelligent body can pass PointNet The visual features of the point cloud data in the interactive environment are extracted, and the point clouds of objects of different categories are inferred as segmented point clouds according to the point cloud classification decoded from the visual features.

[0049] Preferably, in step 2 of the above method, the articulated object manipulation task is encoded into a linear time logic formula in the following manner, including:

[0050] The articulated object manipulation task is encoded as a set of atomic propositions Linear time logic formula created , linear time logic formula The semantics of is defined as:

[0051] ;

[0052] in, Indicates the definition symbol; For atomic propositions; and They are subtask formulas composed of atomic propositions and operators respectively; , They are standard Boolean operators representing negation and conjunction operations respectively; , are time operators representing until and next respectively; the semantics of linear time logic formulas is in the truth value sequence Explain on represents the true value of the i-th label, ;exist When the truth sequence satisfy Recorded as ;

[0053] The progress of the articulated object manipulation task is checked at each moment using the encoded linear time logic formula through the linear time logic series in the following manner, including:

[0054] For a given linear time logic formula and a truth sequence , i = 0 ~ n, in time step ,for ,if , then the linear time logic series , otherwise the series ; Operator With current tag and linear time logic formula As input for each step, it outputs a checked linear time logic formula to indicate the task that the dexterous arm-hand robot is going to complete;

[0055] The current linear time logic formula is encoded into a linear time logic representation using the Transformer model in the following manner, including:

[0056] Given a linear time logic formula Generated Input ,in Represents an operator or proposition, input Through word embedding Preprocessing is to preprocess the input ,in, is a real vector space, is the batch size, Yes Input Length, is the model dimension of the Transformer model, and then, the preprocessing input With frequency-based positional embedding Add together to get a one-dimensional embedded input , linear time logic representation The global calculation process is expressed as:

[0057] ;

[0058] ;

[0059] ;

[0060] ;

[0061] Among them, MSA is the multi-head self-attention module in the Transformer model; LN is the regularization layer in the Transformer model; MLP is the fully connected feedforward layer in the Transformer model; is the linear time logic representation obtained after encoding.

[0062] Preferably, in step 3 of the above method, a contact planner is established by confidence learning in the following manner, wherein the contact planner extracts point cloud features from the segmented point cloud obtained in step 1, predicts the optimal contact point of the articulated object operation task from the point cloud features, and then optimizes the prediction of the contact point using the operation success rate, including:

[0063] Building a contact planner :A contact planner consisting of a PointNet++ and an operability confidence module connected in sequence is used. PointNet++ extracts visual features from the segmented point cloud, and the operability confidence module maps the extracted visual features into the operability score of the articulated object point cloud, converts the articulated object point cloud into the corresponding confidence map, and selects the front with the highest operability score based on the confidence map. P points, and PThe positions of the points are averaged to obtain the maximum availability point observation, and the maximum availability point observation is used as additional observation information. This additional observation information Used as input to the diffusion decision model to guide the end effector of the robot arm interact with articulated objects;

[0064] Set the maximum affordance point reward As an environmental reward Part of the end effector of the manipulator of the dexterous arm-hand robot Observation of the maximum affordance point of interaction Maximize the functional part close to the articulated object, maximum affordance point reward for:

[0065] ;

[0066] Optimizing the contact planner : Based on success rate To update the scale, update and optimize the contact planner as follows:

[0067] ;

[0068] in, represents the updated and optimized contact planner; represents the contact planner before updating and optimization; The end effector of the manipulator of the dexterous arm-hand robot contact points; Manipulation of articulated objects by a dexterous arm-hand robot with a fixed number of steps The corresponding success rate; represents the articulated object i.

[0069] Preferably, in step 4 of the above method, the intelligent agent controlling the dexterous arm-hand robot uses the online diffusion decision model to output actions and interact with the environment, and optimizes the decision of the diffusion decision model according to the sampled data, including:

[0070] Step 41, establish an online diffusion decision model:

[0071] The forward noise addition process of the diffusion decision model Modeled as:

[0072] ;

[0073] in, is a Lipschitz function; Represents a certain data at time step The state at the moment is subsequently given a specific meaning. ; It is a standard Brownian motion; Represents the increment of the following parameter, an infinitely small interval; is a noise schedule that uses forward diffusion time steps Inject a series of monotonically increasing fixed noises;

[0074] The inverse de-noising process of the diffusion decision model Modeled as:

[0075] ;

[0076] in, Represents the generated data of the forward noise addition process The marginal density function of Indicates that at time step The generated data of the forward noise addition process The state distribution of represents reverse Brownian motion; Indicates the noise plan The square of As the generated data of the inverse denoising process The distribution score of ,Will Rewritten as:

[0077] ;

[0078] The data Replace with action , , ;

[0079] The forward denoising process and the reverse denoising process of the diffusion decision model are rewritten as and ;in, From a series of forward noise strategy distribution It can be deduced that when t=0, is the distribution of input strategies, when t=T, is a standard Gaussian distribution; From a series of inverse denoising strategies distribution It is derived from represents the time step in the inverse de-noising process - The generated data of the inverse denoising process The state distribution of

[0080] Step 42, online diffusion decision model output action and environment interaction: When the intelligent agent controlling the dexterous arm-hand robot interacts with the environment, by executing The inverse de-noising process Get the exact solution , the process is as follows:

[0081] ;

[0082] in, represents the weight between noise and original data in the kth step; represents the action of step k; represents the cumulative weight from step 1 to step k; represents the noise estimator; , is the extracted visual feature, is a linear time logic representation, is the maximum availability point observation; k represents the k-th iteration; represents a random vector sampled from a multidimensional standard normal distribution, Represents a normal distribution, and the action of step k is The data obtained from interacting with the environment is added to the experience pool In addition, ,in is a discrete interval The step length, represents a set of positive integers;

[0083] Step 43, online diffusion decision model update optimization: setting noise estimator The loss function is:

[0084] ;

[0085] in, represents mathematical expectation; represents the experience pool;

[0086] Actions that interact with the environment Follow the steps below to update:

[0087] ;

[0088] in, is the learning rate; Indicates action Find the gradient descent; Indicates action The cost function of Use weight The parameterized value function has the following update process:

[0089] ;

[0090] in, and Respectively and The corresponding value at the next moment; is the reward function, which is in the form of:

[0091] ;

[0092] in, It is an environmental reward; is a truth sequence Satisfies the linear time logic formula Additional rewards for: The real state of the environment; is a label function where , is viewed as a set of event detectors, when the atomic proposition Triggered when present in the environment, it allows the dexterous arm-hand robot to determine whether a linear temporal logic formula is satisfied.

[0093] The embodiment of the present invention further provides a processing device, including:

[0094] at least one memory for storing one or more programs;

[0095] At least one processor can execute one or more programs stored in the memory, and when the one or more programs are executed by the processor, the processor can implement the above method.

[0096] The embodiment of the present invention further provides a readable storage medium storing a computer program, which can implement the above method when the computer program is executed by a processor.

[0097] In summary, it can be seen that the optimization method of the embodiment of the present invention enhances the diffusion strategy by using linear time logic representation and confidence learning, thereby improving the learning efficiency and generalization of dexterous arm-hand robot operations. Specifically, the present invention understands the task semantics through linear time logic representation, designs a contact planner to determine the optimal contact points, and then generalizes these contact points to various categories through the diffusion strategy. Based on the interaction data, the present invention further updates the decision of the diffusion strategy model from the perspective of action optimization to overcome the limitations of the traditional diffusion strategy model. Compared with traditional skill learning methods, the present invention is superior to most existing methods in terms of operational ability, generalization performance, transfer reasoning and robustness.

[0098] In order to more clearly demonstrate the technical solution and technical effects provided by the present invention, the solution provided by the embodiment of the present invention is described in detail with specific embodiments below.

[0099] Example 1

[0100] This embodiment provides a method for optimizing the operation performance of a dexterous arm-hand robot's articulated object, which is a generalization method for dexterous arm-hand robot's operation based on a diffusion strategy. Its architecture is as follows: Figure 2 As shown, the linear temporal logic representation (i.e., LTL representation) and the contact planner are combined with the diffusion strategy to improve the learning efficiency of the agent and ensure its generalization ability.

[0101] The LTL representation is obtained by encoding a linear temporal logic formula (i.e., LTL formula) through a Transformer model, and aims to extract the task semantics of the corresponding linear temporal logic formula, thereby improving the learning efficiency of the dexterous arm-hand robot.

[0102] The contact planner guides the dexterous arm-hand robot to interact at the optimal contact point of the articulated object from the perspective of confidence learning, and optimizes the predicted contact point of the planner through the success rate of the operation, thereby achieving efficient operation of the articulated object and laying the foundation for its generalization ability.

[0103] The diffusion strategy model is an online decision-making algorithm under the reinforcement learning framework. It outputs appropriate interactive actions based on observation information and implements algorithm updates and optimization (Action improvement) based on sampling, thereby ensuring that the dexterous arm-hand robot has good generalization.

[0104] The process of the above optimization method is as follows Figure 1 As shown, the specific steps include:

[0105] Step 1, constructing the point cloud information of the dexterous arm-hand robot in the task environment, that is, the environment point cloud dataset, completing the finger point cloud of the dexterous arm-hand robot with the environment point cloud dataset, and segmenting the environment point cloud dataset after completing the finger point cloud of the dexterous arm-hand robot to obtain the segmented point cloud;

[0106] Step 2: Encode the articulated object manipulation task of the dexterous arm-hand robot into an LTL formula, and use the LTL formula to check the progress of the articulated object manipulation task through the LTL series at each moment, and encode the LTL formula at this time into an LTL representation through the Transformer model;

[0107] Step 3: Establish a contact planner from the perspective of confidence learning, extract point cloud features based on the segmented point cloud obtained in step 1, and predict the best contact point. At the same time, optimize the prediction of the contact point in combination with the operation success rate.

[0108] Step 4, enabling the intelligent agent controlling the dexterous arm-hand robot to sample the interaction between the online diffusion decision model output action and the environment, and optimizing the decision of the diffusion decision model according to the sampled data;

[0109] Step 5, repeating the above steps 1 to 4 until the training of the intelligent agent for controlling the dexterous arm-hand robot is completed;

[0110] Step 6: Use the trained intelligent agent controller to control the dexterous arm-hand robot to perform the operation task of the articulated object.

[0111] Furthermore, in step 1 of the above method, point cloud information of the dexterous arm-hand robot in the task environment, i.e., an environmental point cloud dataset, is constructed in the following manner, the finger point cloud of the dexterous arm-hand robot is complemented by the environmental point cloud dataset, and the environmental point cloud dataset after the finger point cloud of the dexterous arm-hand robot is complemented is segmented to obtain a segmented point cloud, specifically:

[0112] By controlling the intelligent agent of the dexterous arm-hand robot to perform random actions in the simulation environment, an environmental point cloud dataset is constructed, which includes: the functional part of the articulated object, the rest of the articulated object, the dexterous hand and the robotic arm;

[0113] The imagined point cloud is used to complete the occluded finger point cloud of the dexterous arm-hand robot. Specifically, the potential posture of each finger is calculated by forward kinematics according to the kinematic model and joint positions of the dexterous arm-hand robot. Then, the imagined point cloud of the corresponding finger is synthesized by sampling points from the grid connected by each finger using the potential posture. The finger point cloud of the dexterous arm-hand robot in the environmental point cloud dataset is completed by the synthesized imagined point cloud of the finger.

[0114] Pre-training with the idea of ​​segmenting objects , so that the intelligent agent controlling the dexterous arm-hand robot can The extracted visual features make basic inferences about the point clouds of objects of different categories, and the inferred point clouds of objects of different categories are used as segmentation point clouds. The visualization of the environment point cloud segmentation and the robot finger point cloud completion is shown in the figure below. Figure 3 shown.

[0115] Furthermore, in step 2 of the above method, the complex articulated object manipulation task is encoded into an LTL formula as follows: an LTL formula consists of a set of atomic propositions When creating, LTL formula The semantics of is defined as:

[0116] ;

[0117] in, Indicates the definition symbol, is an atomic proposition, and is a subtask formula consisting of atomic propositions and operators; , are standard Boolean operators, , They represent negation and conjunction operations respectively. , is the operator, , Represents the time operators until and next respectively. The semantics of LTL formulas is in the truth value sequence Explain on represents the true value of the i-th label, ;exist When the true value sequence satisfy Recorded as ;

[0118] The way to check LTL formulas using LTL series is as follows: given an LTL formula and a truth sequence , at time step ,for ,if , then the LTL series , otherwise the series . Operator Based on LTL formula and the current label As input at each time step, it outputs the checked LTL formula to indicate the task that the dexterous arm-hand robot is to complete.

[0119] The way to encode the LTL formula into LTL representation using the Transformer model is as follows: given a LTL formula Generated Input ,in Represents an operator or proposition, input Through word embedding Preprocessing is to preprocess the input ,in, is a real vector space, is the batch size, Yes Input Length, is the model dimension of the Transformer model, and then, the preprocessing input With frequency-based positional embedding Add together to get a one-dimensional embedded input , the global calculation process of LTL representation can be expressed as:

[0120] ;

[0121] ;

[0122] ;

[0123] ;

[0124] Among them, MSA is the multi-head self-attention module in the Transformer model, LN is the regularization layer in the Transformer model, and MLP is the fully connected feedforward layer in the Transformer model. is the encoded LTL representation.

[0125] Furthermore, in step 3 of the above method, a contact planner is established from the perspective of confidence learning in the following manner, point cloud features are extracted based on the segmented point cloud obtained in step 1, and the optimal contact point is predicted, and the prediction of the contact point is optimized in combination with the operation success rate, specifically:

[0126] Building a contact planner The contact planner is mainly composed of PointNet++ and an actionable affordance module. PointNet++ extracts visual features from the segmented point cloud, while the actionable affordance module further maps the visual features into the actionability score of the object point cloud, thereby converting the object point cloud into a corresponding confidence map. Based on the confidence map, the front end with the highest actionable score is selected. P points and average their positions to obtain the maximum-affordance point observation (MPO). Then, MPO is used as additional observation information. , this additional observation information Used as input to a diffuse decision model to guide the end effector of the robot arm Interact with articulated objects to achieve effective generalization. In addition, the maximum affordance point reward (max-affordance point reward, hereinafter referred to as ),for:

[0127] ;

[0128] Maximum Availability Point Reward As an environmental reward part to enable the end effector Interactive Get as close as possible to the functional part of the articulated object.

[0129] Optimizing the contact planner : Based on success rate To update the scale, the update and optimization of the contact planner are designed using the following method:

[0130] ;

[0131] in, represents the updated and optimized contact planner; represents the contact planner before updating and optimization; The end effector of the manipulator of the dexterous arm-hand robot contact points; Manipulation of articulated objects by a dexterous arm-hand robot with a fixed number of steps The corresponding success rate; represents the articulated object i.

[0132] Furthermore, in step 4 of the above method, the intelligent agent controlling the dexterous arm-hand robot uses the online diffusion decision model to output actions and interact with the environment, and optimizes the decision of the diffusion decision model according to the sampled data, which specifically includes the steps of establishing an online diffusion decision model, interacting with the environment, and updating and optimizing, specifically:

[0133] Step 41, establish an online diffusion decision model: The visualization of the forward noise addition process and the reverse noise removal process of the diffusion decision model is as follows: Figure 4 As shown in the figure, the forward noise addition process of the diffusion decision model It can be modeled as:

[0134] ;

[0135] in, is the Lipschitz function; Represents a certain data at time step The state at the moment is subsequently given a specific meaning. ; It is a standard Brownian motion; Represents the increment of the following parameter, an infinitely small interval; is a noise schedule that uses forward diffusion time steps Injects a series of monotonically increasing fixed noise.

[0136] Inverse de-noising process of diffusion decision model It can be modeled as:

[0137] ;

[0138] in, Represents the generated data of the forward noise addition process The marginal density function of Indicates that at time step The generated data of the forward noise addition process The state distribution of represents reverse Brownian motion; Indicates the noise plan By constructing a noise estimator As the generated data of the inverse denoising process The distribution score of , can be rewritten as:

[0139] ;

[0140] The data Replace with action , , , the forward denoising process and the reverse denoising process of the diffusion decision model can be rewritten as and .therefore, The policy distribution can be added from a series of forward noise It can be deduced that when t=0, is the distribution of input strategies, when t=T, is a standard Gaussian distribution. Similarly, Can be distributed from a series of inverse denoising strategies obtained in , and when hour, ; represents the time step in the inverse de-noising process - The generated data of the inverse denoising process The state distribution of .

[0141] Step 42, online diffusion decision model output action and environment interaction: When the intelligent agent of the dexterous arm-hand robot interacts with the environment, it can be executed The inverse de-noising process To obtain the exact solution, the process is as follows:

[0142] ;

[0143] in, represents the weight between noise and original data in the kth step; represents the action of step k; represents the cumulative weight from step 1 to step k; represents the noise estimator; , is the extracted visual feature, is a linear time logic representation, is the maximum availability point observation; k represents the k-th iteration; represents a random vector sampled from a multidimensional standard normal distribution, Represents a normal distribution, and the action of step k is The data obtained from interacting with the environment is added to the experience pool In addition, ,in is a discrete interval The step length, Represents the set of positive integers.

[0144] Step 43, update optimization of the online diffusion decision model: The architecture of the interaction and update of the online diffusion decision model is shown as follows: Figure 5 As shown, the loss function of the noise estimator can be designed as follows:

[0145] ;

[0146] in, represents mathematical expectation; represents the experience pool;

[0147] Actions that interact with the environment can be updated as follows:

[0148] ;

[0149] in, is the learning rate; Indicates action Find the gradient descent; Indicates action The cost function of Use weight The parameterized value function has the following update process:

[0150] ;

[0151] in, and Respectively and The corresponding value at the next moment; is the reward function, which has the following form:

[0152] ;

[0153] in, It is an environmental reward; is a truth sequence Meeting the LTL formula Additional rewards for: The real state of the environment; middle, is a label function, is viewed as a set of event detectors, when the atomic proposition Triggered when it appears in the environment, it allows the dexterous arm-hand robot to determine whether a linear time logic formula is satisfied. The following is the pseudo code of the dexterous arm-hand robot's articulated object manipulation performance optimization method based on the online diffusion decision model.

[0154]

[0155]

[0156]

[0157] The optimization method of the present invention can solve the problem of generalized performance of the robot in operating the same or different types of articulated objects, can improve the scope of applicable articulated objects, and improve the articulated object operation performance of the dexterous arm-hand robot.

[0158] In summary, the method of the embodiment of the present invention understands the task semantics and improves learning efficiency through LTL representation, builds a contact planner based on confidence learning to determine the optimal contact point of the robot, and then optimizes the diffusion strategy of the diffusion decision model to generalize these interactions to different categories of articulated objects. Based on the interaction data, the present invention further updates the decision of the diffusion strategy from the perspective of motion optimization to overcome the limitations of the traditional diffusion strategy. Compared with traditional skill learning methods, the present invention is superior to most existing methods in terms of operational ability, generalization performance, transfer reasoning and robustness.

[0159] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0160] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with the technical field within the technical scope disclosed in the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims. The information disclosed in the background technology section of this article is only intended to deepen the understanding of the overall background technology of the present invention, and should not be regarded as an admission or in any form that the information constitutes prior art known to those skilled in the art.

Claims

1. A method for optimizing the manipulation performance of articulated objects of a dexterous arm-hand robot, characterized in that: include: Step 1: construct an environmental point cloud dataset of the environment where the dexterous arm-hand robot's articulated object manipulation task is located, complete the finger point cloud of the dexterous arm-hand robot, and segment the environmental point cloud dataset to obtain a segmented point cloud; Step 2, encoding the articulated object manipulation task into a linear time logic formula, checking the progress of the articulated object manipulation task at each moment by using the encoded linear time logic formula through the linear time logic series, and encoding the linear time logic formula at the current moment into a linear time logic representation by using the Transformer model; Step 3, establishing a contact planner through confidence learning, wherein the contact planner extracts point cloud features from the segmented point cloud obtained in step 1, predicts the optimal contact point of the articulated object operation task from the point cloud features, and then optimizes the prediction of the contact point using the operation success rate; Step 4, enabling the intelligent agent controlling the dexterous arm-hand robot to sample the interaction between the online diffusion decision model output action and the environment, and optimizing the decision of the diffusion decision model according to the sampled data; Step 5, repeating steps 1 to 4 until the training of the intelligent agent for controlling the dexterous arm-hand robot is completed; Step 6: Use the trained intelligent agent to control the dexterous arm-hand robot to perform the task of manipulating articulated objects.

2. The method for optimizing the manipulation performance of articulated objects of a dexterous arm-hand robot according to claim 1, characterized in that: In the step 1, an environmental point cloud dataset of the environment where the dexterous arm-hand robot's articulated object manipulation task is located is constructed in the following manner, the finger point cloud of the dexterous arm-hand robot is completed, and the environmental point cloud dataset is segmented to obtain a segmented point cloud, including: Controlling the intelligent agent of the dexterous arm-hand robot to randomly act in a simulation environment to construct an environmental point cloud dataset, the environmental point cloud dataset including: point clouds of functional parts of the articulated object, point clouds of other parts of the articulated object, point clouds of the dexterous hand of the dexterous arm-hand robot and point clouds of the robotic arm; According to the robot kinematic model and joint positions, the potential posture of each finger of the dexterous hand of the dexterous arm-hand robot is calculated by forward kinematics, and the potential posture is used to sample points from the grid connected by each finger to synthesize the imaginary point cloud of the corresponding finger, and the imaginary point cloud of the corresponding finger is used to complete the corresponding finger point cloud of the dexterous hand of the dexterous arm-hand robot when the monocular camera in the environmental point cloud data set is blocked; Pre-training PointNetPN by segmenting objects using the constructed environment point cloud dataset , so that the intelligent body can use PointNetPN The visual features of the point cloud data in the interactive environment are extracted, and the point clouds of objects of different categories are inferred as segmented point clouds according to the point cloud classification decoded from the visual features.

3. The method for optimizing the manipulation performance of articulated objects of a dexterous arm-hand robot according to claim 1 or 2, characterized in that: In step 2, the articulated object manipulation task is encoded into a linear time logic formula in the following manner, including: The articulated object manipulation task is encoded as a set of atomic propositions Linear time logic formula created , linear time logic formula The semantics of is defined as: ; in, Indicates the definition symbol; For atomic propositions; and They are subtask formulas composed of atomic propositions and operators respectively; , They are standard Boolean operators representing negation and conjunction operations respectively; , are time operators representing until and next respectively; the semantics of linear time logic formulas is in the truth value sequence Explain on represents the true value of the i-th label, ;exist When the truth sequence satisfy Recorded as ; The progress of the articulated object manipulation task is checked at each moment using the encoded linear time logic formula through the linear time logic series in the following manner, including: For a given linear time logic formula and a truth sequence , i = 0 ~ n, in time step ,for ,if , then the linear time logic series , otherwise the series ; Operator With current tag and linear time logic formula As input for each time step, it outputs a checked linear time logic formula to indicate the task that the dexterous arm-hand robot is going to complete; The current linear time logic formula is encoded into a linear time logic representation using the Transformer model in the following manner, including: Given a linear time logic formula Generated Input ,in , Represents an operator or proposition, input Through word embedding Preprocessing is to preprocess the input ,in, is a real vector space, is the batch size, Yes Input Length, is the model dimension of the Transformer model, and then, the preprocessing input With frequency-based positional embedding Add together to get a one-dimensional embedded input , linear time logic representation The global calculation process is expressed as: ； ； ； ； Among them, MSA is the multi-head self-attention module in the Transformer model; LN is the regularization layer in the Transformer model; MLP is the fully connected feed-forward layer in the Transformer model; is the linear temporal logic representation obtained after encoding.

4. The method for optimizing the manipulation performance of articulated objects of a dexterous arm-hand robot according to claim 1 or 2, characterized in that: In step 3, a contact planner is established through confidence learning in the following manner, wherein the contact planner extracts point cloud features from the segmented point cloud obtained in step 1, predicts the optimal contact point of the articulated object operation task from the point cloud features, and then optimizes the prediction of the contact point using the operation success rate, including: Building a contact planner :A contact planner consisting of a PointNet++ and an operability confidence module connected in sequence is used. PointNet++ extracts visual features from the segmented point cloud. The operability confidence module maps the extracted visual features into the operability scores of the functional parts of the articulated object in the segmented point cloud, converts the point cloud corresponding to the functional part of the articulated object into the corresponding confidence map, and selects the front with the highest operability score based on the confidence map. P points, and P The positions of the points are averaged to obtain the maximum availability point observation, and the maximum availability point observation is used as additional observation information. ; Set the maximum affordance point reward As an environmental reward Part of the end effector of the manipulator of the dexterous arm-hand robot Observation of the maximum affordance point of interaction Maximize the functional part close to the articulated object, maximum affordance point reward for: ； Optimizing the contact planner : Based on success rate To update the scale, update and optimize the contact planner as follows: ； in, represents the updated and optimized contact planner; represents the contact planner before updating and optimization; The end effector of the manipulator of the dexterous arm-hand robot contact points; Manipulation of articulated objects by a dexterous arm-hand robot with a fixed number of steps The corresponding success rate; represents the articulated object i.

5. The method for optimizing the manipulation performance of articulated objects of a dexterous arm-hand robot according to claim 4, characterized in that: In step 4, the intelligent agent controlling the dexterous arm-hand robot uses the online diffusion decision model to output actions and interact with the environment, and optimizes the decision of the diffusion decision model according to the sampled data, including: Step 41, establish an online diffusion decision model: The forward noise addition process of the diffusion decision model Modeled as: ； in, is a Lipschitz function; Represents a certain data at time step The state in which one is in, which is then given a specific meaning. ; It is a standard Brownian motion; Represents the increment of the following parameter, an infinitely small interval; is a noise plan that uses forward diffusion time steps Inject a series of monotonically increasing fixed noises; The inverse de-noising process of the diffusion decision model Modeled as: ； in, Represents the generated data of the forward noise addition process The marginal density function of Indicates that at time step The generated data of the forward noise addition process The state distribution of represents reverse Brownian motion; Indicates the noise plan The square of As the generated data of the inverse denoising process The distribution score of ,Will Rewritten as: ； The data Replace with action , , ; The forward denoising process and the reverse denoising process of the diffusion decision model are rewritten as and ;in, From a series of forward noise strategy distribution It can be deduced that when t=0, is the distribution of input strategies, when t=T, is a standard Gaussian distribution; From a series of inverse denoising strategies distribution It is derived from represents the time step in the inverse de-noising process - The generated data of the inverse denoising process The state distribution of Step 42, online diffusion decision model output action and environment interaction: When the intelligent agent controlling the dexterous arm-hand robot interacts with the environment, by executing The inverse de-noising process Get the exact solution , the process is as follows: ； in, represents the weight between noise and original data in the kth step; represents the action of step k; represents the cumulative weight from step 1 to step k; represents the noise estimator; , is the extracted visual feature, is a linear time logic representation, is the maximum availability point observation; k represents the k-th iteration; represents a random vector sampled from a multidimensional standard normal distribution, Represents a normal distribution, and the action of step k is The data obtained from interacting with the environment is added to the experience pool In addition, ,in is a discrete interval The step length, represents a set of positive integers; Step 43, online diffusion decision model update optimization: setting the noise estimator The loss function is: ； in, represents mathematical expectation; represents the experience pool; Actions that interact with the environment Follow the steps below to update: ； in, is the learning rate; Indicates action Find the gradient descent; Indicates action The cost function of Use weight The parameterized value function has the following update process: ； in, and Respectively and The corresponding value at the next moment; is the reward function, which is in the form of: ； in, It is an environmental reward; is a truth sequence Satisfies the linear time logic formula Additional rewards for: The real state of the environment; middle, is a label function, is viewed as a set of event detectors, when the atomic proposition Triggered when present in the environment, it allows the dexterous arm-hand robot to determine whether a linear temporal logic formula is satisfied.

6. A processing device, characterized in that: include: at least one memory for storing one or more programs; At least one processor can execute one or more programs stored in the memory, and when the one or more programs are executed by the processor, the processor can implement the method described in any one of claims 1 to 5.

7. A readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 can be implemented.

Images