Robotic grasping control method and system based on successful set parameterized goal

Abstract

The application belongs to the technical field of robot intelligent control and decision, and specifically discloses a robot grasping control method and system based on a success set parameterized target, which comprises the following steps: constructing a success set parameterized target of a robot grasping task, and defining a success judgment predicate; collecting teaching data and interaction data to construct an offline data set, training a conditional diffusion model to obtain a candidate action generator, and generating a candidate action sequence based on a given observation history, a state history and the success set parameterized target; using a pre-trained double Q network value evaluation model to score the value of the candidate action sequence; executing a first action instruction of an optimal candidate action sequence, updating the environment state, regenerating the candidate action sequence for rolling horizon closed-loop control, and stopping until the success judgment predicate is satisfied, thereby completing the robot grasping control. The application solves the problems of difficult expression of a target, difficult coverage of multiple solutions, difficult learning of sparse rewards and insufficient robustness of open-loop planning in mechanical arm grasping.

Description

Technical Field

[0001] This invention belongs to the field of robot intelligent control and decision-making technology, specifically relating to a robot grasping control method and system based on a successful set of parameterized targets. Background Technology

[0002] While the traditional "perception-planning-control" hierarchical pipeline widely used in grasping systems has a clear structure and strong interpretability, it has significant limitations in practical applications. First, due to multiple constraints such as joint limitations, dynamic constraints, end-effector posture requirements, and collision avoidance, the target pose estimated by the perception module often becomes unreachable or conflicting, leading to planning failure or unstable alternative trajectories. Second, grasping tasks inherently have multiple solutions; the same object often has multiple feasible approach directions, gripping postures, and motion paths. If the task is simply defined as a single end-effector pose target, it is necessary to rely on a large number of enumerations and manual rules for filtering, which is inefficient. In addition, during physical execution, disturbances such as contact friction, elastic deformation, object slippage or micro-motion, and perception noise often cause the open-loop planning results, even if theoretically feasible, to fail in closed-loop execution due to model errors. The system lacks the ability to make real-time decisions and adjustments to cope with uncertainties. Meanwhile, existing data-driven methods also have their shortcomings: imitation learning, while stable in training, is susceptible to distribution shifts, has poor adaptability to states not covered by the training set, and cannot utilize task feedback for proactive optimization; reinforcement learning, while possessing the potential for closed-loop self-improvement through reward signals, typically only provides sparse success / failure rewards for grasping tasks, resulting in high exploration costs and slow convergence. Furthermore, the grasping action space is inherently multimodal, while common policy parameterization methods based on Gaussian distributions tend to output "compromise" actions, making it difficult to effectively express and utilize this multimodal characteristic, thus limiting the final grasping success rate. Summary of the Invention

[0003] The purpose of this invention is to solve the problems of difficulty in target representation, difficulty in covering multiple solutions, difficulty in learning sparse rewards, and insufficient robustness of open-loop programming in robotic arm grasping. It proposes a robot grasping control method and system based on parameterized targets of success sets.

[0004] The technical solution of the present invention is as follows: Firstly, a robot grasping control method based on a successfully set parameterized target, comprising the following steps:

[0005] Construct a parameterized objective for the success set of robot grasping tasks, and define a success discrimination predicate that includes position tolerance sub-conditions, proximity direction cone constraint sub-conditions, attitude tolerance sub-conditions, and closure and lift result property sub-conditions.

[0006] An offline dataset is constructed by collecting teaching data and interaction data. The offline data is organized at the granularity of transition tuples and aligned with successful discrimination predicates. The reward signal is calculated through the successful discrimination predicates.

[0007] A conditional diffusion model is trained using an offline dataset to obtain a candidate action generator, and a candidate action sequence is generated based on a given observation history, state history, and success set parameterized target.

[0008] A pre-trained double-Q network value evaluation model is used to score the value of candidate action sequences;

[0009] The candidate action sequences are sorted according to the value score. The first action instruction of the optimal candidate action sequence is executed, the environmental state is updated, and a new candidate action sequence is generated for rolling time-domain closed-loop control until the success judgment predicate is met, thus completing the robot grasping control.

[0010] As a preferred option, the position tolerance sub-condition uses the relative displacement of the end reference point in the object coordinate system as a constraint, and gives an allowable spatial neighborhood, extending single-point alignment to feasible region alignment. The corresponding formula is:

[0011]

[0012] in, This represents the position vector of the end effector reference point in the world coordinate system or base coordinate system at the current moment. This represents the position vector of the target object's centroid or the preset grab point in the same coordinate system at the current moment. This represents a positive definite weight matrix, used to define the geometry of the location tolerance space, enabling weighted control of errors in different dimensions. This indicates the location tolerance threshold.

[0013] As a preferred option, the approach direction cone constraint sub-condition defines the angle constraint between the desired end approach axis unit vector and the current end approach axis, expanding the search space from a single line to a cone region. The corresponding formula is:

[0014]

[0015] in, This indicates that the desired endpoint is a unit vector close to the axis. This indicates that the end is currently approaching the axis. Represents the cosine function. This represents the half-angle of a cone.

[0016] Preferably, the attitude tolerance subcondition is represented by quaternion error, the deviation is measured by the rotation angle and a threshold is set, and the rotational degree of freedom about the object's axis of symmetry is ignored. The corresponding formula is:

[0017]

[0018]

[0019] in, Indicates the rotation angle. Represents the cosine function. Indicates the rotation angle error threshold. The quaternion representing the current attitude of the end effector. This represents the quaternion representing the desired target attitude of the task. This represents the attitude error quaternion, used to quantify the three-dimensional rotational difference between the current end effector coordinate system and the target grasping pose coordinate system. For attitude error quaternions The real part.

[0020] Preferably, the closing and lifting result conditions include the intersection of three logical criteria: the gripper closing judgment, the object's center of gravity height exceeding the lifting threshold judgment, and the judgment that the gripper closing state and the object's center of gravity height exceeding the lifting threshold state are stable and continuously exceeding the time-series threshold.

[0021] As a preferred option, the training objective of the conditional diffusion model is noise regression loss. :

[0022]

[0023] in, Indicates the offline dataset The state in ,Target Original action sequence and the number of diffusion steps for random sampling and Gaussian noise Calculate the mathematical expectation. Represents the original action sequence go through The intermediate state action sequence obtained after adding noise has the following characteristics: , This represents the predefined noise scheduling parameters in the diffusion model. This represents the guiding condition input for the diffusion model.

[0024] As a preferred option, the dual-Q network value evaluation model is trained by minimizing the temporal difference error, and the formula for calculating the target value during training is:

[0025]

[0026] in, Indicates the target value. This represents sparse or semi-sparse rewards triggered by the success set criterion. Indicates the discount factor. Represents the state at a given next time step. and target Under the conditions, for the current strategy Sampled actions Calculate the mathematical expectation. Indicates the first Individual action value network Indicates the execution of the current action. After that, the environment transitions to the state of the next moment. Indicates the state at the next moment. Below, candidate actions are predicted or sampled by the policy network. This represents the entropy regularization coefficient.

[0027] As a preferred method, the specific approach for scoring the value of candidate action sequences using a pre-trained double-Q network value evaluation model is as follows:

[0028] Evaluate the payoff of the first action or the entire sequence of candidate actions:

[0029]

[0030] in, Indicates the first The final value score of the candidate action sequence is used as the basis for subsequent ranking and execution. Indicates the first pre-training A dual-Q network valuation model provides a conservative value estimate by taking the minimum value. This indicates the environmental state input for the robot in the current cycle. Indicates the first The first action instruction in the candidate action sequence. Indicates the success set parameter;

[0031] Calculate sequence value aggregation:

[0032]

[0033] in, Discount factor This power is used to compensate for the time decay of the predictive value for future moments. This represents the future number predicted based on the action sequence. State estimation at each time step Indicates the first The first candidate action sequence in the group Action instructions for each time step Indicates the temporal length of the candidate action sequence;

[0034] The value score is obtained by selecting the first action or the entire segment of the candidate action sequence as the highest value, or by aggregating the sequence values.

[0035] Preferably, the method further includes:

[0036] In sparse reward scenarios, a target relabeling mechanism is used to reconstruct offline or online samples, mapping the actual achieved state to a substitute target. And recalculate the reward:

[0037]

[0038]

[0039] in, This represents the target mapping function, used to extract and convert the actual physical state reached by the robot into parameters in the target space. Indicates the current time step The subsequent future time index is used to select the actual state achieved in the subsequent trajectory as the target for relabeling. Indicates based on alternative goals The recalculated immediate reward for relabeling. The success-determining predicate is used to determine whether the alternative target is being replaced. Next, the state at the next moment Does the constraint condition for the success set satisfy? Indicates an indicator function, when The value is 1 if true, and 0 otherwise.

[0040] Thus constructing enhanced samples The dual-Q network value assessment model will be updated subsequently. Indicates the current state The actual grabbing action command is executed. This indicates that in the reconstructed enhanced sample, at time... The corresponding corrected reward signal.

[0041] The beneficial effects of this invention are:

[0042] This invention reduces the failure rate caused by unreachability and constraint conflicts through successful set modeling; it naturally covers multiple solutions and multi-peak action distributions through diffusion candidate generation; it improves robustness under contact perturbations / noise through critic ranking loop closure; it significantly improves sample efficiency under sparse rewards through target relabeling mechanism; and it suppresses the accumulation of long-term time-domain errors by rolling temporal execution, thereby improving online stability.

[0043] Secondly, a robot grasping control system based on a successfully set parameterized target includes:

[0044] The first module is used to construct a success set parameterized target for robot grasping tasks, defining success discrimination predicates including position tolerance sub-conditions, proximity direction cone constraint sub-conditions, attitude tolerance sub-conditions, and closure and lift result property sub-conditions.

[0045] The second module is used to collect teaching data and interaction data to build an offline dataset. The offline data is organized at the granularity of transition tuples and aligned with the successful discrimination predicate. The reward signal is calculated through the successful discrimination predicate.

[0046] The third module is used to train a conditional diffusion model using an offline dataset to obtain a candidate action generator, and to generate a candidate action sequence based on a given observation history, state history and success set parameterized target.

[0047] The fourth module is used to score the value of candidate action sequences using a pre-trained double-Q network value evaluation model.

[0048] The fifth module is used to sort the candidate action sequences according to the value score, execute the first action instruction of the optimal candidate action sequence, update the environmental state, regenerate the candidate action sequence for rolling time-domain closed-loop control, until the success judgment predicate is met, and complete the robot grasping control. Attached Figure Description

[0049] Figure 1 The diagram shows a flowchart of a robot grasping control method based on a success set parameterized target.

[0050] Figure 2 The diagram shows the overall framework of a robot grasping control method based on a success set parameterized target.

[0051] Figure 3 The diagram shown is a successful set of parameterized targets (position tolerance box / cone approach / attitude tolerance / lifting conditions).

[0052] Figure 4 The diagram shows the forward diffusion and reverse denoising processes of the conditional diffusion model.

[0053] Figure 5 The image shows a diagram illustrating target relabeling under the successful set target.

[0054] Figure 6 The diagram shows the closed-loop execution flow of candidate action generation and value ranking. Detailed Implementation

[0055] Exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be understood that the embodiments shown and described in the drawings are merely exemplary and are intended to illustrate the principles and spirit of the invention, and are not intended to limit the scope of the invention.

[0056] Example 1:

[0057] like Figure 1 and Figure 2 As shown, a robot grasping control method based on a successful set of parameterized targets includes the following steps:

[0058] S1. Construct a parameterized objective for the success set of robot grasping tasks, and define a success discrimination predicate that includes position tolerance sub-conditions, proximity direction cone constraint sub-conditions, attitude tolerance sub-conditions, and closure and lift result property sub-conditions.

[0059] S2. Collect teaching data and interaction data to construct an offline dataset. The offline data is organized at the granularity of transition tuples and aligned with the successful discrimination predicate. The reward signal is calculated through the successful discrimination predicate.

[0060] S3. Train the conditional diffusion model using an offline dataset to obtain a candidate action generator, and generate a candidate action sequence based on the given observation history, state history, and success set parameterized target.

[0061] S4. Use a pre-trained double-Q network value evaluation model to score the value of candidate action sequences;

[0062] S5. Sort the candidate action sequences according to the value score, execute the first action instruction of the optimal candidate action sequence, update the environmental state, regenerate the candidate action sequence for rolling time-domain closed-loop control until the success judgment predicate is met, and complete the robot grasping control.

[0063] In this embodiment, step S1 specifically includes:

[0064] The grasping control is modeled as a Markov decision process. ,in, Represents a set of states, which have states. This includes joint angles / velocities, end-effector pose, object pose, gripper state, and visual features. Represents a set of actions, containing actions. This includes joint velocity / torque or end-effector pose increments; The state transition probability matrix (or state transition model) represents the probability of transitioning from the current state to the current state. Next action Then, the environment transitions to the next state. The probability distribution of is usually denoted as . ; This represents a sparse or semi-sparse reward triggered by the success set criterion. Indicates a discount. The target condition is introduced in the form of... , , ,in This represents the target variable (in this invention, the success set parameter).

[0065] This invention transforms the final state of a grasping task from a traditional "fixed pose point" into a "set of states satisfying specific geometric and physical constraints." This is achieved by targeting... Perform four-dimensional parameterization to construct a robust success discrimination predicate. .

[0066] Expand the "task success" criterion from a single endpoint threshold to a set of criteria:

[0067]

[0068] in, For a predicate to be successfully identified, the following conditions must be met:

[0069]

[0070] This representation is used simultaneously for offline dataset successful annotation, reward construction, HER target relabeling interface, and target condition input for the critic.

[0071] like Figure 3 As shown, the target parameters Combination of the following sub-conditions:

[0072] 1) Location tolerance:

[0073] Robot hardware has repetitive positioning errors, and the object center output by the visual perception module often has centimeter-level noise. Forcing "point-to-point" overlap will cause the controller to oscillate frequently near the endpoint or fail to judge due to slight deviations.

[0074] By using the relative displacement of the end reference point (gripper center / fingertip midpoint) in the object coordinate system as a constraint, and giving an allowable spatial neighborhood (box / ellipsoid / cylindrical, etc.), "single-point alignment" is extended to "feasible region alignment". This definition allows the control strategy to select paths with lower potential energy or easier obstacle avoidance within the tolerance range, significantly improving the fault tolerance and robustness of the controller.

[0075] Specifically, define the displacement vector of the end reference point (such as the gripper center) in the object coordinate system relative to the object's centroid or the preset gripping point. Introducing norm-based Permissible space area:

[0076]

[0077] In engineering practice, Corresponding to the spherical region, and Corresponding to a cuboid tolerance box. Among them, It represents the current system status, including real-time information such as the robot arm joint angles, end effector pose, and object pose. It is the position vector of the end effector reference point (such as the center of the gripper or the midpoint of the fingertip) in the world coordinate system or base coordinate system at the current moment. It is the position vector of the target object's centroid or the preset grab point in the same coordinate system at the current moment. This represents a positive definite weight matrix, used to define the geometry of the position tolerance space (such as the axis length and orientation of an ellipsoid), enabling weighted control of errors in different dimensions. It is the displacement vector of the end reference point relative to the target grab point, used to characterize the degree of spatial alignment; It is the position tolerance threshold, which is the maximum physical distance that the endpoint can deviate from the target point.

[0078] 2) Approach direction cone constraint:

[0079] For cylindrical or rotationally symmetric objects, there are countless possible grasping approaches. Traditional single-point definitions can only force the specification of one direction, which often becomes unreachable due to environmental obstacles or singularities in the robotic arm.

[0080] Define the desired terminal vector as close to the axis as a unit vector. (Usually determined by the object's geometric principal axes or normal vectors) and the current proximity axis of the end point. The angle between them. The success condition is defined as:

[0081]

[0082] in, The cone half-angle physically characterizes the range of directional deflection allowed centered on the desired axis. The larger the angle, the looser the constraint on the approach direction. Centered on the desired approach axis, a directional cone constraint is introduced, and the set of allowed approach directions is characterized by a half-angle threshold to reduce the risk of collision and the probability of failure caused by ungraspable directions.

[0083] This definition method expands the search space from "a line" to "a cone," providing a physical solution set basis for generating diverse candidate actions for the diffusion strategy, which greatly reduces the difficulty of obstacle avoidance planning.

[0084] 3) Attitude tolerance, taking quaternion error as an example:

[0085] In grasping tasks, the rotation of the gripper around the object's axis of symmetry (such as rotating around the central axis when grasping a water cup) usually does not affect the grasping success rate.

[0086] Representing end-effector attitude error using quaternions By rotating the angle The deviation is measured by the current attitude quaternion of the end effector. The decision criterion is... .in, It is the current attitude quaternion of the end effector; It is the expected target attitude quaternion set by the task; The attitude error quaternion is used to quantify the three-dimensional rotational difference between the current end effector coordinate system and the target grasping pose coordinate system. It is a scalar rotation angle error obtained by calculating the real part of the quaternion; This is the rotation angle error threshold, which allows for a small range of rotational deviations around the axis during gripping. For attitude error quaternions The real part is obtained by extracting the real part of the attitude error quaternion. This method can simplify complex three-dimensional rotational deviations into intuitive scalar rotation angles. This enables rapid determination of the accuracy of the grasping posture.

[0087] By explicitly defining attitude tolerance, the system can automatically ignore freedom constraints that are not critical to the task, effectively solving the problem of "multiple solutions leading to a single peak in the policy" and enabling the policy to learn a more generalized action distribution.

[0088] 4) Closure and Lifting Result Conditions

[0089] Geometric alignment does not equate to physical success in grabbing. In sparse reward scenarios, relying solely on geometric metrics can easily induce strategies to engage in "touch and withdraw" score-farming behavior. This dimension encompasses the intersection of three logical criteria:

[0090] ① Closure determination: Define the gripper state Boolean value Ensure that the gripper has executed the closing command.

[0091] ② Lifting height: It determines whether the height of the object's center of gravity exceeds a threshold to verify whether it is truly gripped rather than a false contact. It is the current height of the target object in the vertical direction (Z-axis). It is the lower limit threshold of the lifting height, used as the physical boundary line to distinguish between "merely touching" and "successfully lifting the object".

[0092] ③ Timing maintenance: The above state must remain stable for a period of time to filter out transient perceptual noise. The duration of continuous observation after satisfying all geometric constraints. It is a temporal stability threshold to ensure that successful capture is not due to momentary misjudgment caused by perceptual fluctuations.

[0093] This definition provides the most reliable baseline for HER target relabeling, ensuring the trained value function... It truly reflects the expected benefits of successfully manipulating an object.

[0094] Final success judgment predicate This can be represented as the logical conjunction of the subconditions: multiple subsystems can be combined using "AND / OR": This set-based modeling can significantly reduce the probability of planning failure caused by "single point unreachability" and explicitly accommodate multiple solutions.

[0095] In this embodiment, step S2 specifically involves: data is processed at the transition level. ,in, Indicates time state, For the action to be performed, For instant rewards, The parameters for successfully setting the target set, For possible additional information (such as contact status, etc.). To be consistent with the HER interface, it is organized using achieved_goal / desired_goal: , And by successfully identifying the predicate Calculate the reward: .

[0096] In this embodiment, step S3 uses the trained conditional diffusion model to generate candidate action sequences in the action sequence space:

[0097]

[0098] in, Indicates from the current moment Starting with a length of Candidate action sequences, This indicates the starting action instruction in the action sequence. This indicates the last action instruction in an action sequence. Denotes the high-dimensional real space to which the action sequence belongs, where For time domain length, For the dimension of the action space;

[0099] Diffusion condition input:

[0100]

[0101] in, Indicates time The diffusion-guided feature vector is used to guide the model in generating actions that meet the requirements of the current environment. This indicates a conditional encoder network used to transform historical observations and target information into high-dimensional features. Indicates from time arrive The observation history sequence (such as visual features or sensor data). Indicates from time arrive The robot's state history sequence (such as joint angles or end pose).

[0102] The forward and reverse diffusion processes in the conditional diffusion model are as follows: Figure 4 As shown, let Representing an action sequence, the forward diffusion (noise addition) process is as follows:

[0103]

[0104] in, This represents the single-step noise transfer probability distribution during the diffusion process. This represents a multivariate normal distribution (Gaussian distribution). Indicates the first The noise variance scheduling value of each step determines the intensity of the noise added in each step. This indicates the state of the action sequence in the previous diffusion step (when noise is relatively low). Represents the identity matrix, used to define the covariance structure of multidimensional Gaussian noise;

[0105] Equivalent closed form:

[0106]

[0107]

[0108]

[0109] in, Indicates from the original sequence Directly calculate the first Marginal distribution of the step-noise state, Indicates up to the The cumulative scaling factor up to the step reflects the degree of preservation of the original signal. Indicates the first The signal retention coefficient of the step is defined as follows: , Indicates the current diffusion step index. Indicates the first The basic retention factor of the step.

[0110] The back diffusion (conditional denoising generation) process specifically involves: predicting the output noise of the diffusion network. . use Formal generation:

[0111]

[0112]

[0113] in, The diffusion model is represented in the first... The denoised distribution mean vector predicted step by step guides the direction of regression of the action sequence from the noisy state to the true distribution. This represents a multivariate normal distribution (Gaussian distribution), used to describe the probabilistic characteristics of the sampling process. This refers to the guiding condition input, specifically the target parameterized by observation history, state history, and success set in this invention. Extracted feature representation, This represents the step variance parameter in the backdiffusion process, used to introduce an appropriate amount of random perturbation during the denoising sampling process to maintain the diversity of generated actions.

[0114] In this embodiment, the training objective of the conditional diffusion model is noise regression loss. :

[0115]

[0116] in, Indicates the offline dataset The state in ,Target Original action sequence and the number of diffusion steps for random sampling and Gaussian noise Calculate the mathematical expectation; This represents the true values ​​of the original action sequence extracted from the dataset (i.e., noise-free expert actions or successful interaction trajectories). Indicates the distribution from the standard normal distribution Randomly sampled Gaussian noise is used to simulate the noise addition process during the forward diffusion process; Represents the original action sequence go through The intermediate state action sequence obtained after adding noise has the following characteristics: This represents the predefined noise schedule parameter in the diffusion model, used to control the noise schedule in the first... The original signal is retained during each diffusion step. proportion, This represents the discrete-time step index in the diffusion process, typically taking values ​​in the range of... This represents a diffuse neural model (typically a U-Net or MLP architecture) to be trained, whose input is a noisy sequence. diffusion steps and guiding conditions The output is the predicted noise; The guiding condition inputs for the diffusion model, in this invention, are parameterized by the current observation history, state history, and success set. It is jointly encoded.

[0117] In this embodiment, the maximum entropy objective of the pre-trained target conditional value evaluation network is:

[0118]

[0119] in, Indicates adherence to the current strategy The generated trajectory Calculate the mathematical expectation. Discount factor The power of a power is used to decay the value of rewards at different future time steps, reflecting the principle of "nearer is better than farther". Indicates the target Under constraints, state Execute action The instant reward signal obtained Indicates the policy in the state Information entropy, used to measure the dispersion of action distribution or exploratory ability, This represents the entropy regularization coefficient (or temperature parameter), used to balance the weighting relationship between "pursuing high rewards" and "maintaining strong exploration".

[0120] Uses dual Q: The target value is:

[0121]

[0122] in, Indicates the target value. This represents sparse or semi-sparse rewards triggered by the success set criterion. This represents the discount factor, and its value typically ranges from 100 to 1000. Between these, the weights are used to balance current immediate rewards and future long-term benefits. Represents the state at a given next time step. and target Under the conditions, for the current strategy Sampled actions Calculate the mathematical expectation. Indicates the first This invention employs a dual-Q network (Q network) to create an action value network. By taking the minimum value, the error of overestimating value can be suppressed. Indicates the execution of the current action. After that, the environment transitions to the state of the next moment. Indicates the state at the next moment. Below, candidate actions are predicted or sampled by the policy network. This represents the entropy regularization coefficient, used to control the exploratory nature of the strategy; The larger the size, the more the strategy tends to be diversified in its exploration.

[0123] critic loss for:

[0124]

[0125] In engineering implementation, it is possible to "train only the critic and not explicitly train the actor probability density" to achieve closed-loop decision-making through candidate selection and ranking; this is more stable under sparse rewards and complex action distributions.

[0126] In this embodiment, as Figure 5 As shown, the specific relabeling of the target under the successful set objective is as follows: under sparse rewards, positive samples are scarce. HER maps the "actually achieved state" to alternative targets. And recalculate the rewards:

[0127]

[0128]

[0129] Thus, enhanced samples are constructed:

[0130]

[0131] Due to the successful collection of predicates in this invention and The organization is naturally consistent and can be directly embedded into the HER process.

[0132] In this embodiment, the closed-loop execution process of candidate action generation and value ranking is as follows: Figure 6 As shown, in each control cycle, samples are obtained. Group candidate action sequence ;

[0133] Evaluate the initial move or overall benefit for each group of candidates:

[0134]

[0135] Or sequence value aggregation:

[0136]

[0137] Select the largest:

[0138]

[0139] Before execution only Step (often taken) Then, rolling replanning. Rolling time-domain closed-loop control executes the previous step, resampling, and re-evaluation, which can effectively suppress the accumulation of long-term time-series errors.

[0140] Example 2:

[0141] Based on Embodiment 1, this embodiment of the invention provides a robot grasping control system based on a success set parameterized target, which can be used to implement the robot grasping control method based on a success set parameterized target as described in the foregoing embodiments. The system includes:

[0142] The first module is used to construct a successful set parameterized target for robot grasping tasks, defining a success discrimination predicate that includes position tolerance sub-conditions, proximity direction cone constraint sub-conditions, attitude tolerance sub-conditions, and closure and lift result property sub-conditions.

[0143] The second module is used to collect teaching data and interaction data to build an offline dataset. The offline data is organized at the granularity of transition tuples and aligned with the successful discrimination predicate. The reward signal is calculated through the successful discrimination predicate.

[0144] The third module is used to train a conditional diffusion model using an offline dataset to obtain a candidate action generator, and to generate a candidate action sequence based on a given observation history, state history and success set parameterized target.

[0145] The fourth module is used to score the value of candidate action sequences using a pre-trained double-Q network value evaluation model.

[0146] The fifth module is used to sort the candidate action sequences according to the value score, execute the first action instruction of the optimal candidate action sequence, update the environmental state, regenerate the candidate action sequence for rolling time-domain closed-loop control, until the success judgment predicate is met, and complete the robot grasping control.

[0147] According to embodiments of the present invention, the present invention also provides an electronic device, a readable storage medium, and a computer program product.

[0148] In an exemplary embodiment, the electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the robot grasping control method based on a success set parameterized target as described in Embodiment 1 above.

[0149] In an exemplary embodiment, the readable storage medium may be a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the robot grasping control method based on a success set parameterized target as described in Embodiment 1 above.

[0150] In an exemplary embodiment, the computer program product includes a computer program that, when executed by a processor, implements the robot grasping control method based on a success set parameterized target as described in Embodiment 1 above.

[0151] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0152] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0153] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0154] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0155] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0156] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

Claims

1. A robot grasping control method based on a success set parameterized target, characterized in that, Includes the following steps: Construct a parameterized objective for the success set of robot grasping tasks, and define a success discrimination predicate that includes position tolerance sub-conditions, proximity direction cone constraint sub-conditions, attitude tolerance sub-conditions, and closure and lift result property sub-conditions. An offline dataset is constructed by collecting teaching data and interaction data. The offline data is organized at the granularity of transition tuples and aligned with successful discrimination predicates. The reward signal is calculated through the successful discrimination predicates. A conditional diffusion model is trained using an offline dataset to obtain a candidate action generator, and a candidate action sequence is generated based on a given observation history, state history, and success set parameterized target. A pre-trained double-Q network value evaluation model is used to score the value of candidate action sequences; The candidate action sequences are sorted according to the value score. The first action instruction of the optimal candidate action sequence is executed, the environmental state is updated, and the candidate action sequence is regenerated for rolling time-domain closed-loop control until the success judgment predicate is met, and the robot grasping control is completed. The position tolerance subcondition, constrained by the relative displacement of the end reference point in the object coordinate system, and given an allowable spatial neighborhood, extends single-point alignment to feasible region alignment. The corresponding formula is: in, This represents the position vector of the end effector reference point in the world coordinate system or base coordinate system at the current moment. This represents the position vector of the target object's centroid or the preset grab point in the same coordinate system at the current moment. This represents a positive definite weight matrix, used to define the geometry of the location tolerance space, enabling weighted control of errors in different dimensions. Indicates the location tolerance threshold; The cone constraint sub-condition for approach direction defines the angle constraint between the desired terminal approach axis unit vector and the current terminal approach axis, expanding the search space from a single line to a cone region. The corresponding formula is: in, This indicates that the desired endpoint is a unit vector close to the axis. This indicates that the end is currently approaching the axis. Represents the cosine function. Indicates the half-angle of a cone; The attitude tolerance subcondition is represented by quaternion errors. The deviation is measured by the rotation angle, and a threshold is set. Rotational degrees of freedom around the object's axis of symmetry are ignored. The corresponding formula is: in, Indicates the rotation angle. Represents the cosine function. Indicates the rotation angle error threshold. The quaternion representing the current attitude of the end effector. This represents the quaternion representing the desired target attitude of the task. This represents the attitude error quaternion, used to quantify the three-dimensional rotational difference between the current end effector coordinate system and the target grasping pose coordinate system. For attitude error quaternions The real part; The closing and lifting result conditions include the intersection of three logical criteria: the gripper closure determination, the object's center of gravity height exceeding the lifting threshold determination, and the determination that the gripper closure state and the object's center of gravity height exceeding the lifting threshold state are stable and continuously exceeding the time-series threshold.

2. The robot grasping control method based on a successful set parameterized target according to claim 1, characterized in that, The training objective of the conditional diffusion model is noise regression loss. : in, Indicates the offline dataset The state in ,Target Original action sequence and the number of diffusion steps for random sampling and Gaussian noise Calculate the mathematical expectation. Represents the original action sequence go through The intermediate state action sequence obtained after adding noise has the following characteristics: , This represents the predefined noise scheduling parameters in the diffusion model. This represents the guiding condition input for the diffusion model.

3. The robot grasping control method based on a successful set parameterized target according to claim 1, characterized in that, The dual-Q network value evaluation model is trained by minimizing the temporal difference error. The formula for calculating the training target value is as follows: in, Indicates the target value. This represents sparse or semi-sparse rewards triggered by the success set criterion. Indicates the discount factor. Represents the state at a given next time step. and target Under the conditions, for the current strategy Sampled actions Calculate the mathematical expectation. Indicates the first Individual action value network Indicates the execution of the current action. After that, the environment transitions to the state of the next moment. Indicates the state at the next moment. Below, candidate actions are predicted or sampled by the policy network. This represents the entropy regularization coefficient.

4. The robot grasping control method based on a successful set parameterized target according to claim 1, characterized in that, The method for scoring the value of candidate action sequences using a pre-trained double-Q network value evaluation model is as follows: Evaluate the payoff of the first action or the entire sequence of candidate actions: in, Indicates the first The final value score of the candidate action sequence is used as the basis for subsequent ranking and execution. Indicates the first pre-training A dual-Q network valuation model provides a conservative value estimate by taking the minimum value. This indicates the environmental state input for the robot in the current cycle. Indicates the first The first action instruction in the candidate action sequence. Indicates the success set parameter; Calculate sequence value aggregation: in, Discount factor This power is used to compensate for the time decay of the predictive value for future moments. This represents the future number predicted based on the action sequence. State estimation at each time step Indicates the first The first candidate action sequence in the group Action instructions for each time step Indicates the temporal length of the candidate action sequence; The value score is obtained by selecting the first action or the entire segment of the candidate action sequence as the highest value, or by aggregating the sequence values.

5. The robot grasping control method based on a successful set parameterized target according to claim 1, characterized in that, The method further includes: In sparse reward scenarios, a target relabeling mechanism is used to reconstruct offline or online samples, mapping the actual achieved state to a substitute target. And recalculate the reward: in, This represents the target mapping function, used to extract and convert the actual physical state reached by the robot into parameters in the target space. Indicates the current time step The subsequent future time index is used to select the actual state achieved in the subsequent trajectory as the target for relabeling. Indicates based on alternative goals The recalculated immediate reward for relabeling. The success-determining predicate is used to determine whether the alternative target is being replaced. Next, the state at the next moment Does the constraint condition for the success set satisfy? Indicates an indicator function, when The value is 1 if true, and 0 otherwise. Thus constructing enhanced samples The dual-Q network value assessment model will be updated subsequently. Indicates the current state The actual grabbing action command is executed. This indicates that in the reconstructed enhanced sample, at time... The corresponding corrected reward signal.

6. A robot grasping control system based on a successful set of parameterized targets, characterized in that, include: The first module is used to construct a success set parameterized target for robot grasping tasks, defining success discrimination predicates including position tolerance sub-conditions, proximity direction cone constraint sub-conditions, attitude tolerance sub-conditions, and closure and lift result property sub-conditions. The second module is used to collect teaching data and interaction data to build an offline dataset. The offline data is organized at the granularity of transition tuples and aligned with the successful discrimination predicate. The reward signal is calculated through the successful discrimination predicate. The third module is used to train a conditional diffusion model using an offline dataset to obtain a candidate action generator, and to generate a candidate action sequence based on a given observation history, state history and success set parameterized target. The fourth module is used to score the value of candidate action sequences using a pre-trained double-Q network value evaluation model. The fifth module is used to sort the candidate action sequences according to the value score, execute the first action instruction of the optimal candidate action sequence, update the environmental state, regenerate the candidate action sequence for rolling time-domain closed-loop control until the successful discrimination predicate is met, and complete the robot grasping control. The position tolerance subcondition, constrained by the relative displacement of the end reference point in the object coordinate system, and given an allowable spatial neighborhood, extends single-point alignment to feasible region alignment. The corresponding formula is: in, This represents the position vector of the end effector reference point in the world coordinate system or base coordinate system at the current moment. This represents the position vector of the target object's centroid or the preset grab point in the same coordinate system at the current moment. This represents a positive definite weight matrix, used to define the geometry of the location tolerance space, enabling weighted control of errors in different dimensions. Indicates the location tolerance threshold; The cone constraint sub-condition for approach direction defines the angle constraint between the desired terminal approach axis unit vector and the current terminal approach axis, expanding the search space from a single line to a cone region. The corresponding formula is: in, This indicates that the desired endpoint is a unit vector close to the axis. This indicates that the end is currently approaching the axis. Represents the cosine function. Indicates the half-angle of a cone; The attitude tolerance subcondition is represented by quaternion errors. The deviation is measured by the rotation angle, and a threshold is set. Rotational degrees of freedom around the object's axis of symmetry are ignored. The corresponding formula is: in, Indicates the rotation angle. Represents the cosine function. Indicates the rotation angle error threshold. The quaternion representing the current attitude of the end effector. This represents the quaternion representing the desired target attitude of the task. This represents the attitude error quaternion, used to quantify the three-dimensional rotational difference between the current end effector coordinate system and the target grasping pose coordinate system. For attitude error quaternions The real part; The closing and lifting result conditions include the intersection of three logical criteria: the gripper closure determination, the object's center of gravity height exceeding the lifting threshold determination, and the determination that the gripper closure state and the object's center of gravity height exceeding the lifting threshold state are stable and continuously exceeding the time-series threshold.

Images