Control method and device for robot end effector

Abstract

The application discloses a control method and device of a robot end effector. The method comprises the following steps: receiving a to-be-operated task and a corresponding point cloud diagram of the end effector at a target time; using a prediction model in a pre-trained action model to predict the to-be-operated task and the point cloud diagram, and obtaining an output result, wherein the output result is used to represent a three-dimensional bottleneck representation of a motion trajectory of the end effector; using a strategy model in the action model to analyze the output result, and generating an action sequence of the end effector; and controlling the end effector to perform an operation corresponding to the action sequence. The method provided in the application at least solves the technical problem that, in the related art, an action sequence of an end effector is mainly generated based on a large amount of modeling data, and the action sequence generation process is relatively complex.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a control method and device of a robot end effector. BACKGROUND

[0002] The robot end effector (dexterous hand) brings great challenges to operation tasks due to its high-dimensional action space. Traditional control methods usually use optimization methods, which are usually focused on a narrow task range, such as grasping. Recently, end-to-end methods have made significant progress in dexterous manipulation. Reinforcement learning has become a popular method, which generates data through carefully designed reward functions and continuous interaction with the environment, constantly updates the learning data set and policy network, and finally enables the dexterous hand to complete tasks such as playing the piano, object reorientation in hand, and dynamic catch. Although the method based on reinforcement learning has achieved certain effect, there are still some challenges. In order to be deployed in the real world, the corresponding Sim-to-Real (from simulation to reality) method must be designed to solve the problem of policy adaptation of real robots. In addition, the dependence of reinforcement learning algorithm on reward design limits its applicability in specific scenarios. Imitation learning is another widely used method, and the introduction of diffusion models enables imitation learning algorithms to model more rich robot trajectory information. However, there are still the following key problems: imitation learning methods rely on a large amount of teleoperation high-quality data, and there are great difficulties in modeling high-dimensional and multi-peak action sequences. The existing imitation learning paradigm is mainly based on 2D image data to predict the future trajectory of the robot, but the robot trajectory is rich in three-dimensional spatial information, which generally needs to be represented using point cloud and other modal data. SUMMARY

[0003] The embodiments of the present application provide a control method and device of a robot end effector, which at least solve the technical problem in the related art that the action sequence of the end effector is mainly generated based on a large amount of modeling data, resulting in a complex action sequence generation process.

[0004] According to an aspect of an embodiment of the present application, a control method of a robot end effector is provided, including: receiving a to-be-operated task and a corresponding point cloud map of an end effector at a target time; using a prediction model in a pre-trained action model to predict the to-be-operated task and the point cloud map, to obtain an output result, wherein the output result is used to represent a three-dimensional bottleneck representation of a motion trajectory of the end effector; using a policy model in the action model to analyze the output result, to generate an action sequence of the end effector; and controlling the end effector to perform an operation corresponding to the action sequence.

[0005] Optionally, the action model is determined by: acquiring a first training dataset, wherein the first training dataset includes at least: a point cloud map corresponding to the end-effector and sensor data of the end-effector, and the data in the first training dataset does not contain action labels; training the prediction model using the first training dataset to obtain a trained prediction model, wherein the prediction model includes a bidirectional encoder and a bidirectional decoder; acquiring a second training dataset, wherein the second training dataset includes at least: historical action sequences of the end-effector and motion trajectories of the end-effector; determining conditional input data based on the output of the prediction model; and training the policy model using the second training dataset and the conditional input data to obtain a trained policy model.

[0006] Optionally, training the prediction model using the first training dataset to obtain a trained prediction model includes: encoding the point cloud map corresponding to the end-effector and the sensor data of the end-effector to obtain trajectory vectors; concatenating multiple trajectory vectors in chronological order to obtain a latent space feature trajectory set; sampling multiple fixed-length latent space feature segments from the latent space feature trajectory set, wherein the latent space feature segments contain historical and future parts; masking the future part of the latent space feature segments using a preset mask at a preset ratio to obtain masked latent space feature segments; and training the prediction model using the masked latent space feature segments to obtain a trained prediction model.

[0007] Optionally, the prediction model is trained using the masked latent space feature fragments to obtain a trained prediction model, including: using the bidirectional encoder to predict the masked latent space feature fragments to obtain a three-dimensional bottleneck representation; and using the bidirectional decoder to decode the three-dimensional bottleneck representation and the target latent space feature fragments to obtain a predicted trajectory, wherein the target latent space feature fragments represent that the future portion of the latent space feature fragments is completely masked.

[0008] Optionally, determining the conditional input data based on the output of the prediction model includes: obtaining a fixed-length motion sub-trajectory segment of the end-effector from the second training dataset; extracting features from the historical portion of the motion sub-trajectory segment of the end-effector using a point cloud encoder to obtain a historical vector; and determining the conditional input data based on the historical vector and the output of the prediction model.

[0009] Optionally, training the policy model using the second training dataset and the conditional input data to obtain a trained policy model includes: adding noise to the action sequence corresponding to the motion sub-trajectory segment of the end-effector to obtain a target action sequence; and training the policy model using the target action sequence and the conditional input data to obtain a trained policy model.

[0010] Optionally, the output result is analyzed using the strategy model in the action model to generate the action sequence of the end-effector, including: obtaining the historical three-dimensional representation vector before the target time; determining the input condition data of the target time based on the output result and the historical three-dimensional representation vector; and analyzing the input condition data of the target time using the strategy model to obtain the action sequence of the end-effector.

[0011] According to another aspect of the embodiments of this application, a control device for a robot end effector is also provided, comprising: a receiving module for receiving a task to be operated and a point cloud map corresponding to the end effector at a target time; a prediction module for predicting the task to be operated and the point cloud map using a prediction model in a pre-trained motion model to obtain an output result, wherein the output result is used to represent a three-dimensional bottleneck representation of the motion trajectory of the end effector; a generation module for analyzing the output result using a strategy model in the motion model to generate a motion sequence of the end effector; and an operation module for controlling the end effector to perform an operation corresponding to the motion sequence.

[0012] According to another aspect of the embodiments of this application, a computer device is also provided, including: a memory and a processor, wherein the memory is used to store program instructions; the processor, connected to the memory, is used to execute the control method of the robot end effector described above.

[0013] According to another aspect of the embodiments of this application, a computer program product is also provided, including computer instructions that, when executed by a processor, implement the control method of the robot end effector described above.

[0014] In this embodiment, the process involves receiving a task to be performed and a point cloud map corresponding to the end-effector at a target time; using a prediction model in a pre-trained action model to predict the task to be performed and the point cloud map to obtain an output result, wherein the output result is used to represent the three-dimensional bottleneck representation of the motion trajectory of the end-effector; using a strategy model in the action model to analyze the output result and generate an action sequence for the end-effector; controlling the end-effector to perform operations corresponding to the action sequence. This two-stage process of generating the action sequence—extracting the three-dimensional representation vector of the motion trajectory from the point cloud data and generating the action sequence based on the three-dimensional representation vector of the motion trajectory—reduces the need for a large amount of modeling data, thus simplifying the action sequence generation process. This solves the technical problem in related technologies where the generation of end-effector action sequences is mainly based on a large amount of modeling data, leading to a complex action sequence generation process. Attached Figure Description

[0015] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0016] Figure 1 This is a hardware structure block diagram of a computer terminal for implementing a control method for a robot end effector according to an embodiment of this application;

[0017] Figure 2 This is a flowchart of a control method for a robot end effector according to an embodiment of this application;

[0018] Figure 3 This is a flowchart of an action model training method according to an embodiment of this application;

[0019] Figure 4 This is a flowchart of another control method for a robot end effector according to an embodiment of this application;

[0020] Figure 5 This is a structural diagram of a control device for a robot end effector according to an embodiment of this application. Detailed Implementation

[0021] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0022] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0023] The information collected in this application embodiment is information and data authorized by the user or fully authorized by all parties. The collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data all comply with the relevant laws, regulations and standards of the relevant regions, and necessary confidentiality measures have been taken. It does not violate public order and good morals, and provides corresponding operation entry points for users to choose to authorize or reject the automated decision results. If the user chooses to reject, the process will proceed to the expert decision-making process.

[0024] To better understand the embodiments of this application, the technical terms used in the embodiments of this application are explained as follows:

[0025] Reinforcement learning (RL) is a branch of machine learning that emphasizes learning optimal policies through interaction with the environment. The agent adjusts its behavior by receiving rewards or penalties through a trial-and-error process, aiming to maximize long-term cumulative rewards. Reinforcement learning is widely used in fields such as robot control and autonomous driving.

[0026] Sim-to-Real Transfer refers to the process of transferring a learned strategy from a simulation environment to a real-world environment. Because interaction with the real environment is costly, robot strategy training is typically conducted in a simulation environment. Sim-to-Real aims to reduce the gap between simulation and reality, enabling models trained in simulation to be successfully applied in real-world operations.

[0027] Transformer is a deep learning architecture for processing sequential data, widely used in natural language processing tasks. It captures the relationships between different parts of a sequence through self-attention, enabling efficient parallel computation and handling long-range dependencies. The Transformer architecture is the foundation of many modern natural language processing models, such as BERT and GPT.

[0028] BTE (Bidirectional Transformer Encoder): This is an encoder structure in the Transformer model that employs a bidirectional self-attention mechanism. When processing the input sequence, it considers not only the information flow from beginning to end but also the information flow from end to beginning. This bidirectionality makes BTE more comprehensive in understanding context, giving it a significant advantage, especially in text understanding tasks.

[0029] BTD (Bidirectional Transformer Decoder): BTD is the decoder part of the Transformer architecture, similar to BTE, but it focuses on generating the output sequence in the task. BTD also uses a bidirectional self-attention mechanism, which allows the decoder to refer to past outputs and future context information simultaneously during generation, helping to generate more coherent and accurate sequences.

[0030] DDPM (Denoising Diffusion Probabilistic Model): DDPM is a generative model that generates high-quality data samples through a progressive denoising process. The model starts with highly noisy data, gradually denoises it, and ultimately generates realistic data. DDPM demonstrates superior performance in image generation and other generative tasks, producing high-quality results.

[0031] Diffusion Policy: Diffusion Policy is a reinforcement learning method for policy generation based on a diffusion model. It denoises the action sequence as "noise" in the diffusion process, generating high-quality actions that meet task requirements. Diffusion Policy demonstrates strong generative capabilities in handling complex, long-term tasks, particularly excelling in high-dimensional spaces.

[0032] DDIM (Denoising Diffusion Implicit Models): DDIM is an improved diffusion model. Compared to DDPM, it modifies the inference method of the denoising process, providing more efficient inference capabilities when generating samples. DDIM can generate high-quality samples with fewer denoising steps, typically offering higher generation speeds and making it suitable for tasks requiring real-time generation.

[0033] To address the problems existing in related technologies, this application provides a control method for a robot end effector, which can operate in... Figure 1 The computer terminal shown is explained below.

[0034] The control method for the robot end effector provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware block diagram of a computer terminal for implementing a control method for a robot end effector is shown. Figure 1 As shown, the computer terminal 10 may include one or more processors (shown as 102a, 102b, ..., 102n in the figure) (the processor may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission module 106 for communication functions connected via wired and / or wireless networks. In addition, it may also include: a display, a keyboard, a cursor control device, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, and a BUS bus. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0035] It should be noted that the aforementioned one or more processors and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10. As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0036] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the control method of the robot end effector in this embodiment. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the aforementioned control method of the robot end effector. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0037] The transmission module 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission module 106 includes a network interface controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission module 106 may be a radio frequency (RF) module, used for wireless communication with the Internet.

[0038] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10.

[0039] It should be noted here that, in some optional embodiments, the above... Figure 1 The computer terminal shown may include hardware elements (including circuitry), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. It should be noted that... Figure 1 This is only one instance of a specific particular instance, and is intended to illustrate the types of components that may exist in the aforementioned computer terminal.

[0040] In the above operating environment, this application provides an embodiment of a control method for a robot end effector. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Also, although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than that shown here.

[0041] Figure 2This is a flowchart of a control method for a robot end effector according to an embodiment of this application, such as... Figure 2 As shown, the method includes the following steps:

[0042] Step S202: Receive the task to be operated and the point cloud map corresponding to the terminal execution structure at the target time;

[0043] Step S204: The prediction model in the pre-trained action model is used to predict the task to be operated and the point cloud map to obtain the output result, wherein the output result is used to represent the three-dimensional bottleneck representation of the motion trajectory of the end-effector.

[0044] Step S206: Analyze the output result using the strategy model in the action model to generate the action sequence of the end execution structure; control the end execution structure to perform the operation corresponding to the action sequence.

[0045] Through steps S202 to S206 above, the process involves receiving the task to be operated and the corresponding point cloud map of the end-effector at the target time; using a prediction model in a pre-trained action model to predict the task to be operated and the point cloud map to obtain an output result, wherein the output result is used to represent the three-dimensional bottleneck representation of the motion trajectory of the end-effector; using a strategy model in the action model to analyze the output result and generate an action sequence of the end-effector; controlling the end-effector to perform the operation corresponding to the action sequence. This two-stage process of generating the action sequence—extracting the three-dimensional representation vector of the motion trajectory from the point cloud data and generating the action sequence based on the three-dimensional representation vector of the motion trajectory—reduces the need for a large amount of modeling data, thus simplifying the action sequence generation process. This solves the technical problem that related technologies mainly rely on large amounts of modeling data to generate end-effector action sequences, leading to a complex action sequence generation process. A detailed explanation follows.

[0046] In some embodiments of this application, the action model is determined as follows: A first training dataset is obtained, wherein the first training dataset includes at least: a point cloud map corresponding to the end-effector and sensor data of the end-effector, and the data in the first training dataset does not contain action labels; the prediction model is trained using the first training dataset to obtain a trained prediction model, wherein the prediction model includes a bidirectional encoder and a bidirectional decoder; a second training dataset is obtained, wherein the second training dataset includes at least: a historical action sequence of the end-effector and a motion trajectory of the end-effector; conditional input data is determined based on the output of the prediction model; the policy model is trained using the second training dataset and the conditional input data to obtain a trained policy model.

[0047] In this embodiment, the dynamics of the trajectory are first modeled using a predicted representation (output) learned through spatial representation. This predicted representation is then used as an additional conditional signal for learning the downstream diffusion strategy. In the first stage, a Transformer-based encoder-decoder architecture is employed to learn a spatial representation from a sequence of point cloud features (point cloud map). The network is trained in a self-supervised manner, predicting the trajectory from a masked trajectory view by predicting the masked future state. The masked sequence is encoded as multiple bottleneck markers, which are combined with the masked marker sequence to reconstruct the entire trajectory, thereby compressing the predicted representation of the future state in the spatial representation. In the second stage, a diffusion strategy is trained centrally using a small action-annotated dataset, conditioned on the predicted representation and the encoding of the current point cloud (point cloud map at the target time). This diffusion strategy then generates an executable multi-step action sequence through a denoising process. By integrating the point cloud encoding and the predicted spatial representation learned in the state-only trajectory, trajectory-level and step-level representations are effectively utilized to denoise and obtain the future actions of the dexterous hand.

[0048] It is understood that the control method for the robot end effector proposed in this application involves: first, using an encoder-decoder architecture to model the trajectory in order to extract a three-dimensional representation of the future trajectory (output result). These three-dimensional representations are then used to provide future information for diffusion policy learning.

[0049] In some embodiments of this application, the specific steps for training the prediction model using the first training dataset to obtain the trained prediction model are as follows: Encoding the point cloud map corresponding to the end-effector and the sensor data of the end-effector to obtain trajectory vectors; concatenating multiple trajectory vectors in chronological order to obtain a latent space feature trajectory set; sampling multiple fixed-length latent space feature segments from the latent space feature trajectory set, wherein each latent space feature segment contains a historical part and a future part; masking the future part of the latent space feature segment using a preset mask at a preset ratio to obtain a masked latent space feature segment; training the prediction model using the masked latent space feature segment to obtain the trained prediction model.

[0050] 3D Information Extraction: In the dataset used in the first stage, the trajectory is represented as... ,in, This represents the total number of time steps. Specifically, Indicates time step Point clouds, This represents the corresponding proprioceptive data (sensor data). Here, It refers to the number of points in the point cloud image. It refers to the number of sensors on the robot. It represents the set of real numbers.

[0051] Point clouds are obtained by transforming depth maps from a single depth camera using intrinsic parameters, followed by cropping to exclude points outside the workspace. Propriometry (sensor data) varies depending on the environment, including proprioception for dexterous hands and robotic arms. Notably, action labels are not required at this stage. These point clouds are sparse, downsampled, and do not contain RGB information—typically derived from... or It consists of points. Although the shape of the data is... However, using this input to model trajectories in dexterity tasks remains a challenging problem, especially compared to previous methods that used simpler input modalities.

[0052] Since the goal is to model the entire dexterous hand trajectory, a method is sought that can reduce prediction complexity while maintaining expressive power. Given its excellent performance in point cloud compression and high task success rate on multiple benchmark tasks, the open-source DP3 point cloud encoder is adopted to compress the point cloud map and its self-perception into a 128-dimensional vector (trajectory vector).

[0053]

[0054] In the formula, This indicates that the point cloud encoder represents the trajectory. The process of processing.

[0055] These vectors are concatenated in chronological order to form a set of latent space feature trajectories:

[0056]

[0057] like Figure 3 As shown, in the first stage, a bidirectional Transformer architecture is introduced, consisting of a bidirectional Transformer encoder (BTE) and a bidirectional Transformer decoder (BTD). The encoder processes the training dataset through a linear encoder and a sinusoidal position encoder to generate a bottleneck representation. The decoder then reconstructs the complete trajectory from this bottleneck, including past and future observations.

[0058] By extracting latent space feature trajectory sets Multiple fixed-length latent space feature fragments were sampled. Apply a future mask to each segment. This masks a fixed proportion of the future trajectory. The masked segment is then fed into a bidirectional Transformer encoder (BTE) to generate a predictable 3D bottleneck representation (output):

[0059]

[0060] Here, BTE indicates that the encoder is performing an encoding operation, and M indicates that the mask operation is performing a mask operation.

[0061] The specific process of training the prediction model using the masked latent space feature fragments to obtain the trained prediction model is as follows: the bidirectional encoder is used to predict the masked latent space feature fragments to obtain a three-dimensional bottleneck representation; the bidirectional decoder is used to decode the three-dimensional bottleneck representation and the target latent space feature fragments to obtain the predicted trajectory, wherein the target latent space feature fragments represent the future part of the latent space feature fragments being completely masked.

[0062] Subsequently, the bidirectional Transformer decoder (BTD) receives... Using a fully occluded feature sequence as input, the estimated trajectory is reconstructed:

[0063]

[0064] Here, BTD represents the decoding operation.

[0065] Training loss is defined as the mean squared error (MSE) between the estimated trajectory and the true trajectory:

[0066]

[0067] In this way, the aim is The algorithm simultaneously encodes historical context information and predictions of future observations. In the first stage of training, it uses future point clouds without action labels and self-awareness information—information often ignored in previous implementations. Previous approaches typically relied solely on past states and future actions to train the policy. The output of the first stage... It is a 128-dimensional vector containing a representation of predictive 3D information. This representation will serve as a key condition in the second-stage policy training and will be fused with the representation output by a lightweight encoder as a conditional input for the actions of the reasoning agent.

[0068] In some embodiments of this application, the specific steps for determining the conditional input data based on the output of the prediction model are as follows: obtaining a fixed-length motion sub-trajectory segment of the end-executive structure from the second training dataset; using a point cloud encoder to extract features from the historical portion of the motion sub-trajectory segment of the end-executive structure to obtain a historical vector; and determining the conditional input data based on the historical vector and the output of the prediction model.

[0069] It should be noted that the output results This is achieved by encoding the entire trajectory segment into a 128-dimensional vector. The entire network encoding predictive features is frozen in the second stage to retain its predictive capabilities learned during pre-training. Furthermore, a lightweight point cloud encoder consisting of a 3-layer MLP network is used to extract the sub-trajectory... Historical section Extract a 128-dimensional vector of a historical feature, and the final conditional input data is constructed as follows:

[0070]

[0071] Understandably, during the training process, each trajectory in the second training data is first divided into lengths. The trajectory segments. For each trajectory segment, a shorter sub-trajectory is extracted. and their corresponding action sequences .

[0072] In some embodiments of this application, the specific steps for training the policy model using the second training dataset and the conditional input data to obtain the trained policy model are as follows: adding noise to the action sequence corresponding to the motion sub-trajectory segment of the end-effector to obtain the target action sequence; training the policy model using the target action sequence and the conditional input data to obtain the trained policy model.

[0073] Specifically, by extracting a shorter sub-trajectory and their corresponding action sequences This serves as the training sample for the policy network. The noisy scheduler randomly selects a time step. For action segments Add to Step noise, obtain noise action (Target action sequence). Training loss is defined as:

[0074]

[0075] in, This represents the mean square error. , These represent the weights of the action sequence and the noise, respectively.

[0076] In some embodiments of this application, the specific steps for analyzing the output results using the strategy model in the action model to generate the action sequence of the end-effector are as follows: Figure 4 As shown, the historical three-dimensional representation vector before the target time is obtained; the input condition data of the target time is determined based on the output result and the historical three-dimensional representation vector; the input condition data of the target time is analyzed using the strategy model to obtain the action sequence of the end-execution structure.

[0077] It should be noted that the historical 3D representation vector can be determined based on the point cloud map and sensor data before the target time. The determination method is similar to that of the trajectory vector, and will not be elaborated here.

[0078] To better illustrate the control method of the robot end effector provided in the embodiments of this application, a specific embodiment is described below.

[0079] Benchmark environment for robotic end effectors (dexterous hands): This benchmark involves using a dexterous robotic hand to perform various everyday tasks, such as opening a door or moving an object to a target location. This benchmark highlights the advantages of dexterous hands, especially in tasks such as "spinning a pen," where the wrist is fixed and the pen must be rotated to a target pose using all five fingers. The dexterous hand used in this benchmark has 24 degrees of freedom (DoF). Three tasks are used as examples: spinning a pen (in-hand manipulation), opening a door (hinge-based manipulation), and using a hammer (tool use). These tasks involve motion spaces of 24, 26, and 28 DoF, respectively, reflecting different hand base configurations. For each task, the initial pose of the object is randomly set.

[0080] Training data collection: Existing reinforcement learning methods were used to obtain expert demonstration datasets. For each task, two datasets were collected: a large-scale set of unlabeled trajectories for representation learning (first training dataset), and a small-scale set of action-annotated trajectories for policy training (second training dataset). For low-dimensional one-handed tasks in dexterous hands and gripper-based tasks in Meta-World, 50 unlabeled trajectories were collected for representation training, and 10 labeled trajectories were collected for policy learning. In contrast, for high-dimensional two-handed tasks (action space exceeding 40 degrees of freedom) in Bi-Dexhands benchmarks, 200 unlabeled trajectories and 50 labeled trajectories were collected. Point cloud data was obtained by converting depth maps acquired by a single-view camera into point clouds, then cropping to retain points within the workspace, and downsampling to 512 or 1024 points.

[0081] Model and Training Configuration: During Transformer encoder training, each input sequence was divided into trajectory segments of length 10, with both future and historical observations having a length of 5. The encoder and decoder consisted of 2 and 1 Transformer layers, respectively. Each Transformer layer had an embedding dimension of 128, included 4 attention heads, and a dropout rate of 0.1. The bottleneck output of the Transformer encoder was also 128-dimensional, consistent with the 3D feature dimension for easy addition. The DP3 point cloud encoder had an input channel dimension of 3 and an output dimension of 64. Its parameters were completely frozen during training to avoid representation collapse. The encoded point cloud features were concatenated with the state information projected through an MLP to form a 128-dimensional 3D feature representation. The model was trained for 2000 epochs to ensure convergence, and performance was monitored using a validation set, with checkpoints showing minimum loss functions to prevent overfitting. During training of the second-stage policy model, the historical observation length was set to 3, and the model was configured to predict actions over 3 consecutive steps. The DDIM noise scheduler was used, with 100 steps of denoising during the training phase and only 10 steps during the testing phase. A sampling-based prediction method was also employed to accelerate convergence.

[0082] The model's application process: Taking the pen-rotation task as an example, a simulation environment is first initialized. In space, a dexterous hand with a fixed wrist, a blue pen for manipulation, and a green pen as the target (the target pen's pose is randomized) are initialized. Next, the trained model is loaded. A monocular depth camera acquires a depth map and converts it into a point cloud map, which is then input into a pre-built predictive encoder and a DP3 point cloud encoder. The output features are summed and input into a diffusion strategy to generate the movement of each finger joint. This is then executed in the simulator. After acquiring new point cloud observations, this process is repeated iteratively. Success is determined when the dexterous hand rotates the pen to match the pose of the target pen. Failure is determined if the pen is not rotated to the designated pose within a certain time or if the pen is dropped.

[0083] Figure 5 A control device for a robot end effector is shown, the device comprising:

[0084] The receiving module 60 is used to receive the task to be operated and the point cloud map corresponding to the end execution structure at the target time;

[0085] Prediction module 62 is used to predict the task to be operated and the point cloud map using a prediction model in a pre-trained action model to obtain an output result, wherein the output result is used to represent the three-dimensional bottleneck representation of the motion trajectory of the end-effector.

[0086] The generation module 64 is used to analyze the output result using the strategy model in the action model and generate the action sequence of the end execution structure;

[0087] Operation module 66 is used to control the end execution structure to perform operations corresponding to the action sequence.

[0088] The control device for the aforementioned robot end effector receives the task to be operated and the point cloud map corresponding to the end effector structure at the target time; it uses a prediction model in a pre-trained motion model to predict the task to be operated and the point cloud map to obtain an output result, wherein the output result is used to represent the three-dimensional bottleneck representation of the motion trajectory of the end effector structure; it uses a strategy model in the motion model to analyze the output result and generate an action sequence of the end effector structure; it controls the end effector structure to perform operations corresponding to the action sequence. The action sequence is generated through a two-stage process of extracting the three-dimensional representation vector of the motion trajectory from the point cloud data and generating the action sequence based on the three-dimensional representation vector of the motion trajectory. This reduces the need for a large amount of modeling data, thereby simplifying the action sequence generation process and solving the technical problem that the action sequence generation process is relatively complex due to the fact that related technologies mainly rely on a large amount of modeling data to generate the action sequence of the end effector structure.

[0089] The prediction module 62 includes: a training submodule, used to acquire a first training dataset, wherein the first training dataset includes at least: a point cloud map corresponding to the end-effector and sensor data of the end-effector, and the data in the first training dataset does not contain action labels; to train the prediction model using the first training dataset to obtain a trained prediction model, wherein the prediction model includes a bidirectional encoder and a bidirectional decoder; to acquire a second training dataset, wherein the second training dataset includes at least: a historical action sequence of the end-effector and a motion trajectory of the end-effector; to determine conditional input data based on the output of the prediction model; and to train the policy model using the second training dataset and the conditional input data to obtain a trained policy model.

[0090] The training submodule includes a first training unit and a second training unit. The first training unit is used to train the prediction model using the first training dataset to obtain a trained prediction model. This includes: encoding the point cloud map corresponding to the end-effector and the sensor data of the end-effector to obtain trajectory vectors; concatenating multiple trajectory vectors in chronological order to obtain a latent space feature trajectory set; sampling multiple fixed-length latent space feature segments from the latent space feature trajectory set, wherein each latent space feature segment contains a historical part and a future part; masking the future part of the latent space feature segments using a preset mask at a preset ratio to obtain masked latent space feature segments; and training the prediction model using the masked latent space feature segments to obtain a trained prediction model.

[0091] The first training unit includes: a first training subunit, used to train the prediction model using the masked latent space feature fragments to obtain a trained prediction model, including: using the bidirectional encoder to predict the masked latent space feature fragments to obtain a three-dimensional bottleneck representation; using the bidirectional decoder to decode the three-dimensional bottleneck representation and the target latent space feature fragments to obtain a predicted trajectory, wherein the target latent space feature fragments represent that the future part of the latent space feature fragments is completely masked.

[0092] The second training unit is used to determine conditional input data based on the output of the prediction model, including: obtaining a fixed-length motion sub-trajectory segment of the end-executive structure from the second training dataset; extracting features from the historical part of the motion sub-trajectory segment of the end-executive structure using a point cloud encoder to obtain a historical vector; and determining the conditional input data based on the historical vector and the output of the prediction model.

[0093] The second training unit includes a second training subunit, used to train the policy model using the second training dataset and the conditional input data to obtain a trained policy model, including: adding noise to the action sequence corresponding to the motion sub-trajectory segment of the end-effector to obtain a target action sequence; and using the target action sequence and the conditional input data to train the policy model to obtain a trained policy model.

[0094] The control device for the robot end effector further includes: a generation submodule, used to analyze the output result using the strategy model in the motion model to generate the motion sequence of the end effector structure, including: obtaining the historical three-dimensional representation vector before the target time; determining the input condition data of the target time based on the output result and the historical three-dimensional representation vector; and analyzing the input condition data of the target time using the strategy model to obtain the motion sequence of the end effector structure.

[0095] It should be noted that, Figure 5 The control device of the robot end effector shown is used to perform Figure 2 The control method of the robot end effector shown above also applies to the control device of the robot end effector, and will not be repeated here.

[0096] This application also provides a computer device, including: a memory and a processor, wherein the memory is used to store program instructions; and the processor, connected to the memory, is used to execute the control method of the robot end effector described above.

[0097] This application also provides a computer program product, including computer instructions that, when executed by a processor, implement the steps of the control method for the robot end effector in this application.

[0098] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0099] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0100] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0101] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0102] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0103] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0104] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A control method for a robot end effector, characterized in that, include: Receive the task to be operated and the point cloud map corresponding to the terminal execution structure at the target time; The prediction model in the pre-trained action model is used to predict the task to be performed and the point cloud map to obtain an output result. The output result is used to represent the three-dimensional bottleneck representation of the motion trajectory of the end-effector. The prediction model is determined by: obtaining a first training dataset, wherein the first training dataset includes at least: the point cloud map corresponding to the end-effector and the sensor data of the end-effector, and the data in the first training dataset does not contain action labels; training the prediction model using the first training dataset to obtain a trained prediction model, wherein the prediction model includes a bidirectional encoder and a bidirectional decoder. The prediction model is trained using the first training dataset to obtain a trained prediction model, including: encoding the point cloud map corresponding to the end-effector and the sensor data of the end-effector to obtain trajectory vectors; concatenating multiple trajectory vectors in chronological order to obtain a latent space feature trajectory set; sampling multiple fixed-length latent space feature segments from the latent space feature trajectory set, wherein each latent space feature segment contains a historical part and a future part; masking the future part of the latent space feature segment using a preset mask at a preset ratio to obtain a masked latent space feature segment; and training the prediction model using the masked latent space feature segment to obtain a trained prediction model. The bidirectional encoder is used to predict the latent space feature fragments after the masking process to obtain a three-dimensional bottleneck representation; The output result is analyzed using the strategy model in the action model to generate the action sequence of the terminal execution structure; The terminal execution structure is controlled to perform the operation corresponding to the action sequence.

2. The method according to claim 1, characterized in that, The motion model is determined in the following way: Obtain a second training dataset, wherein the second training dataset includes at least: the historical action sequence of the end-effector and the motion trajectory of the end-effector; Determine the conditional input data based on the output of the prediction model; The policy model is trained using the second training dataset and the conditional input data to obtain the trained policy model.

3. The method according to claim 1, characterized in that, The prediction model is trained using the latent space feature fragments after the masking process, resulting in a trained prediction model, including: The bidirectional decoder is used to decode the three-dimensional bottleneck representation and the target latent space feature fragment to obtain the predicted trajectory, wherein the target latent space feature fragment indicates that the future part of the latent space feature fragment is completely occluded.

4. The method according to claim 2, characterized in that, The conditional input data is determined based on the output of the prediction model, including: Obtain motion sub-trajectory segments of fixed-length end-executive structures from the second training dataset; A point cloud encoder is used to extract features from the historical portion of the motion sub-trajectory segment of the end-effector to obtain a historical vector; The conditional input data is determined based on the historical vector and the output of the prediction model.

5. The method according to claim 4, characterized in that, The policy model is trained using the second training dataset and the conditional input data to obtain a trained policy model, including: Noise is added to the action sequence corresponding to the motion sub-trajectory segment of the end-effector to obtain the target action sequence; The policy model is trained using the target action sequence and the conditional input data to obtain the trained policy model.

6. The method according to claim 1, characterized in that, The output result is analyzed using the strategy model in the action model to generate an action sequence for the final execution structure, including: Obtain the historical three-dimensional representation vector prior to the target time; The input condition data for the target time are determined based on the output results and the historical three-dimensional representation vector. The input condition data at the target time is analyzed using the strategy model to obtain the action sequence of the terminal execution structure.

7. A control device for a robot end effector, characterized in that, include: The receiving module is used to receive the task to be operated and the point cloud map corresponding to the end execution structure at the target time; The prediction module is used to predict the task to be performed and the point cloud map using a prediction model in a pre-trained action model, and obtain an output result. The output result is used to represent the three-dimensional bottleneck representation of the motion trajectory of the end-effector. The prediction model is determined by: acquiring a first training dataset, wherein the first training dataset includes at least: the point cloud map corresponding to the end-effector and the sensor data of the end-effector, and the data in the first training dataset does not contain action labels; training the prediction model using the first training dataset to obtain a trained prediction model, wherein the prediction model includes a bidirectional encoder and a bidirectional decoder. The prediction model is trained using the first training dataset to obtain a trained prediction model, including: encoding the point cloud map corresponding to the end-effector and the sensor data of the end-effector to obtain trajectory vectors; concatenating multiple trajectory vectors in chronological order to obtain a latent space feature trajectory set; sampling multiple fixed-length latent space feature segments from the latent space feature trajectory set, wherein each latent space feature segment contains a historical part and a future part; masking the future part of the latent space feature segment using a preset mask at a preset ratio to obtain a masked latent space feature segment; and training the prediction model using the masked latent space feature segment to obtain a trained prediction model. The bidirectional encoder is used to predict the latent space feature fragments after the masking process to obtain a three-dimensional bottleneck representation; The generation module is used to analyze the output results using the strategy model in the action model and generate the action sequence of the end-execution structure; An operation module is used to control the terminal execution structure to perform operations corresponding to the action sequence.

8. A computer device, characterized in that, include: A memory and a processor, wherein the memory is used to store program instructions; The processor, connected to the memory, is used to execute the control method for the robot end effector according to any one of claims 1 to 6.

9. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the control method for the robot end effector as described in any one of claims 1 to 6.

Images