A task action generation method and device, a robot, electronic equipment and a medium

Abstract

This disclosure provides a task action generation method, apparatus, robot, electronic device, and medium, relating to the field of robot control. The method includes acquiring a task action generation model, which includes at least one sub-network, each sub-network being trained to correspond to at least one task type; determining a target sub-network within the at least one sub-network of the task action generation model based on the matching degree between task data and the at least one sub-network, the task data including at least one of sensor images, joint positions, and verbal commands; and determining a target task action corresponding to the task data through the target sub-network, the target task action being used by the robot to execute the target task corresponding to the task data. This method, through a divide-and-conquer strategy, enables each sub-network to be responsible for generating task actions within its specialized domain, reducing computational costs, eliminating manual intervention, improving adaptability, meeting real-time requirements, and improving the efficiency and accuracy of task action generation.

Description

Technical Field

[0001] This disclosure relates to the field of robot control, and more particularly to a method and apparatus for generating task actions, a robot, electronic equipment, and a medium. Background Technology

[0002] In the field of robot control, embodied intelligence and robot learning technologies are developing rapidly, with robot imitation learning becoming an important approach to achieving efficient operational strategy learning. With the fusion of multimodal information such as vision, speech, and motion, large-scale models are showing significant potential in robot behavior understanding and decision-making. Summary of the Invention

[0003] This disclosure provides a method and apparatus for generating task actions, a robot, an electronic device, and a medium. It proposes a method for efficiently decomposing and jointly modeling multi-task operation behaviors to improve the generalization and specialization performance of robots in complex and diverse task scenarios.

[0004] A first aspect of this disclosure proposes a task action generation method, comprising: acquiring a task action generation model, the task action generation model including at least one sub-network, each sub-network being trained to correspond to at least one task type; determining a target sub-network in at least one sub-network of the task action generation model based on the degree of matching between task data and the sub-network, the task data including at least one of sensor images, joint positions, and language commands; and determining a target task action corresponding to the task data through the target sub-network, the target task action being used by the robot to perform the target task corresponding to the task data.

[0005] In some embodiments of this disclosure, determining a target subnetwork in at least one subnetwork of a task action generation model based on the degree of matching between task data and subnetworks includes: for a first subnetwork in the task action generation model, determining the energy value corresponding to the task data and the first subnetwork through a gating network, wherein the energy value reflects the degree of matching between the task data and the first subnetwork; determining the probability value corresponding to the task data and the first subnetwork based on the energy value, wherein the probability value is negatively correlated with the energy value; and determining the first subnetwork whose probability value meets a preset condition as the target subnetwork.

[0006] In some embodiments of this disclosure, the target task action corresponding to the task data is determined through a target sub-network, including: sampling an initial noisy action sequence in a standard Gaussian distribution; performing iterative denoising processing on the initial noisy action sequence based on the task data to determine the target task action, wherein the target task action includes the joint trajectory of at least one joint of the robot.

[0007] In some embodiments of this disclosure, iterative denoising processing is performed on an initial noisy action sequence based on task data, including: inputting the initial noisy action sequence and task data into the denoising model of the target sub-network to determine the predicted noise corresponding to the initial noisy action sequence; determining a first noisy action sequence based on the predicted noise through denoising sampling processing; performing iterative denoising processing on the first noisy action sequence until the number of iterations reaches a preset number of iterations to determine the target task action.

[0008] In some embodiments of this disclosure, the method further includes: sending a target task action to the robot's controller to control at least one joint of the robot to perform the target task according to the joint trajectory in the target task action.

[0009] In some embodiments of this disclosure, the method further includes: acquiring an initial training dataset, the initial training dataset including at least one set of training data pairs, each training data pair including initial task data and an initial task action corresponding to the initial task data; determining the energy value corresponding to the initial task data and a second sub-network in at least one sub-network through a gating network; updating the buffer corresponding to the second sub-network based on the energy value, the buffer including at least one set of initial training data pairs; and updating the network layer parameters of the second sub-network based on the at least one set of initial training data pairs in the buffer.

[0010] In some embodiments of this disclosure, updating the network layer parameters of the second sub-network based on at least one set of initial training data pairs in the buffer includes: determining at least one set of first training data pairs in the buffer, the first training data pairs including first initial task data and a first initial task action corresponding to the first initial task data; determining a predicted task action based on the first initial task data through the second sub-network; and determining the network layer parameters of the second sub-network based on the predicted task action and the first initial task action corresponding to the first initial task data.

[0011] In some embodiments of this disclosure, the method further includes: inputting initial task data from the initial training dataset into a gating network to determine the probability distribution corresponding to the initial task data; and updating the parameters of the gating network by gradient descent based on an optimized loss function, wherein the optimized loss function includes a reconstruction objective, a specialization objective, and a coverage objective.

[0012] In the above embodiments, a gating network is used to allocate each input task data to the target sub-network in the task action generation model to activate the target sub-network for task action generation. This avoids interference between tasks, achieves decoupling between tasks, reduces the computational cost of the model, improves the efficiency and accuracy of task action generation, and further enhances the system's adaptability and cross-task scalability.

[0013] A second aspect of this disclosure provides a task action generation apparatus configured to execute the task action generation method proposed in the first aspect of this disclosure.

[0014] A third aspect of this disclosure provides a robot configured to perform a task action generation method proposed in a first aspect of this disclosure, or to include a task action generation apparatus proposed in a second aspect of this disclosure.

[0015] A fourth aspect of this disclosure provides an electronic device including: a processor and a memory for storing a computer program capable of running on the processor, wherein the processor, when running the computer program, performs the method described in any embodiment of the first aspect of this disclosure.

[0016] A fifth aspect of this disclosure provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the methods described in any of the embodiments of the first aspect of this disclosure.

[0017] A sixth aspect of this disclosure provides a program product including computer instructions for causing a computer to perform the methods described in any of the embodiments of the first aspect of this disclosure.

[0018] A seventh aspect of this disclosure provides a chip including at least one processor and a communication interface; the communication interface is used to receive signals input to the chip or signals output from the chip, and the processor communicates with the communication interface and implements the method described in any embodiment of the first aspect of this disclosure through logic circuits or executing code instructions.

[0019] In summary, the task action generation method proposed in this disclosure obtains a task action generation model, which includes at least one sub-network, each trained to correspond to at least one task type. Based on the matching degree between task data and the sub-networks, a target sub-network is determined from at least one sub-network of the task action generation model. The task data includes at least one of sensor images, joint positions, and language commands. Through the target sub-network, the target task action corresponding to the task data is determined, and the target task action is used by the robot to perform the target task corresponding to the task data. This disclosure, through a divide-and-conquer strategy, enables each sub-network to be responsible for generating task actions within its professional domain, reducing the computational cost of the model, eliminating human intervention, improving adaptability, meeting real-time requirements, and improving the efficiency and accuracy of task action generation.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0022] Figure 1 This is a flowchart illustrating a task action generation method proposed in an embodiment of this disclosure;

[0023] Figure 2 This is a flowchart illustrating another task action generation method proposed in an embodiment of this disclosure;

[0024] Figure 3 This is a flowchart illustrating another task action generation method proposed in an embodiment of this disclosure;

[0025] Figure 4 This is a flowchart illustrating another task action generation method proposed in an embodiment of this disclosure;

[0026] Figure 5 This is a flowchart illustrating another task action generation method proposed in an embodiment of this disclosure;

[0027] Figure 6 This is a flowchart illustrating another task action generation method proposed in an embodiment of this disclosure;

[0028] Figure 7 This is a flowchart illustrating another task action generation method proposed in an embodiment of this disclosure;

[0029] Figure 8 This is a flowchart illustrating another task action generation method proposed in an embodiment of this disclosure;

[0030] Figure 9 The overall architecture diagram of the method for constructing a robotic arm behavior model based on a hybrid expert network and energy model;

[0031] Figure 10 This is a schematic diagram of the structure of a task action generation device according to an embodiment of the present disclosure;

[0032] Figure 11 This is a schematic diagram of the structure of the electronic device proposed in the embodiments of this disclosure. Detailed Implementation

[0033] Embodiments of this disclosure are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this disclosure, and should not be construed as limiting this disclosure.

[0034] In related fields, robots can be divided into various types such as humanoid robots, robotic arms, wheeled + dual-arm robots, tracked robots, and fixed-base robots. Through remote control, robots can perform related tasks.

[0035] In related technologies, the diffusion model has been introduced into robot operation learning, enabling the modeling and generation of multimodal action distributions and effectively improving the stability and flexibility of complex operation tasks. Subsequently, the Vision-Language-Action (VLA) model further unifies visual perception, language understanding, and action control within a single framework, promoting the development of general robot operation capabilities. However, in multi-task learning scenarios, a single model still suffers from "inter-task interference" and "averaging effects" when simultaneously handling diverse tasks, leading to performance degradation and difficulties in policy convergence.

[0036] Hybrid Expert Networks (MoEs) have been proposed in the field of natural language processing, significantly improving model parameter scale and training efficiency through sparse activation mechanisms. In recent years, the MoE concept has been gradually introduced into robot reinforcement learning and multi-task control to achieve policy diversification and task specialization. For example, in reinforcement learning tasks, MoEs can adapt to heterogeneous task spaces through different expert networks; in robot operation control, MoEs can achieve adaptive fusion of multimodal sensor data. However, they still largely rely on manually dividing expert or task subspaces, lacking automated skill decomposition and dynamic expert selection mechanisms.

[0037] The robotic arm behavior model in related technologies has the following technical problems:

[0038] First, in multi-task imitation learning, when a single model is used to learn multiple different tasks, the gradients of different tasks will interfere with each other due to the sharing of model parameters, leading to conflicts in optimization directions. The model will tend to learn an "average" solution for all tasks, rather than the optimal solution for each task, resulting in a significant drop in performance on a single task, i.e., the model "learns everything but is not good at anything".

[0039] Second, the modeling capability for multimodal action distribution is insufficient. A single task may involve multiple plausible successful strategies (i.e., multimodal action distribution). For example, placing an object can be done in several ways, such as "directly placing" or "gently pushing in." While methods like diffusion models alleviate this problem to some extent, in multi-task datasets, a single model still struggles to clearly represent and maintain all effective modes for all tasks.

[0040] Third, there is a reliance on human prior knowledge for skill classification. Related technologies involve manually classifying datasets into tasks or skills, and then assigning different categories of data to different expert networks. This method heavily depends on human domain knowledge, incurs high annotation costs, and struggles to guarantee the accuracy and optimality of the classification, thus limiting the model's scalability and generalizability.

[0041] Therefore, to address the aforementioned technical challenges, this disclosure proposes a task action generation method that combines the MoE architecture with an Energy-based Model (EBM) gating mechanism. This method achieves dynamic selection of task-related experts and decoupled learning of multi-task actions through a gating network and a task action generation model. While maintaining computational sparsity, the proposed method effectively improves the model's learning efficiency and generalization ability in multi-task scenarios, providing a new technical path for efficient decision-making and autonomous learning in embodied intelligent systems.

[0042] The following describes the techniques used in this disclosure:

[0043] (1) Diffusion Policy

[0044] Diffusion Policy applies a diffusion model to robot imitation learning. This method uses a diffusion process as the policy framework to predict future action sequences through progressive denoising, thus naturally modeling the multimodal action distribution under a task. Its basic process is as follows: Observed images (or point clouds) and historical actions are input into an encoder, and after feature extraction, they are fed into the diffusion model. Control actions are generated through multi-step reverse denoising.

[0045] (2) π0 (Vision-Language-Action Flow Model)

[0046] π0 is a large-scale pre-trained general-purpose robot policy model that combines a diffusion model with a vision-language pre-training architecture (similar to RT-2). This model perceives multi-task information through unified language and visual input and outputs action tokens or joint control signals, exhibiting strong generalization and language understanding capabilities. However, its overall architecture is a monolithic model with severe coupling of internal skill representations, resulting in high inference costs, a lack of modularity, and difficulty in achieving independent invocation or reuse of specific skills.

[0047] (3) ACT (Action Chunking with Transformers)

[0048] ACT employs a Transformer architecture, achieving long-term behavior cloning by predicting action chunks over a future period. This method demonstrates good stability and control frequency in multi-task imitation learning, and can capture long-term dependencies. However, its monolithic structure still faces the problem of gradient conflicts between tasks. When faced with significantly different task types (such as fine grasping versus high-speed throwing), the model tends to adopt a compromise strategy, leading to performance degradation.

[0049] The method proposed in this disclosure first decomposes complex multi-task learning into collaborative learning of multiple sub-networks by introducing a task action generation model. Each sub-network focuses on learning one or more highly relevant sub-skills, thereby fundamentally avoiding direct interference between tasks and improving the overall performance of the model in multi-task scenarios.

[0050] Secondly, by combining MoE with a diffusion model, each sub-network can focus on learning the multimodal action distribution within its skill domain, while the gating network is responsible for allocating task data to the most suitable sub-network. This "divide and conquer" strategy enhances the ability to model the complex multimodal distribution of the entire action space.

[0051] Third, this disclosure employs an energy model to automatically learn the observation distribution of each sub-network, and uses a gating network to automatically allocate data. The entire process eliminates the need for manually labeled skill tags, achieving automated skill decomposition and sub-network specialization, thus improving the method's practicality and scalability.

[0052] The task action generation method proposed in this disclosure can be applied to the field of humanoid robots, and can also be extended to the fields of robotic arms or wheeled + dual-arm robots, etc.

[0053] The task action generation method provided in this application will be described in detail below with reference to the accompanying drawings.

[0054] In some embodiments, the task action generation method proposed in this disclosure can be applied to complex operation scenarios requiring robotic arms to quickly switch between multiple tasks, make intelligent decisions, and execute with high precision. The task action generation method proposed in this disclosure can understand task semantics and automatically select the most suitable sub-network, thereby achieving accurate, stable, and efficient multi-task operations.

[0055] For example, in a home environment, service robots need to continuously perform a variety of tasks, such as picking up a fragile ceramic cup from a table, throwing scattered toys into a storage box, and neatly folding dried clothes. These tasks vary greatly in terms of motion patterns, force requirements, and movement trajectories. By using a task motion generation model that integrates multiple sub-networks to achieve task-adaptive skill invocation, the stability and high success rate of actions can be maintained during continuous execution of multiple tasks.

[0056] For example, in a factory assembly line, robotic arms need to perform vastly different tasks at different stages of a smart manufacturing scenario, such as precision assembly (high precision and low force control), high-speed sorting (high response and anti-interference), and flexible material handling (compliant control).

[0057] Figure 1 This is a flowchart illustrating a task action generation method proposed in an embodiment of this disclosure, as shown below. Figure 1 As shown, the method includes the following steps:

[0058] The task action generation method proposed in this disclosure can be executed by a terminal device, which can be a robot, and the type of robot is not limited in this disclosure.

[0059] Step 101: Obtain the task action generation model.

[0060] In some embodiments, the task action generation model includes at least one sub-network, each sub-network being trained to correspond to at least one task type.

[0061] In some embodiments, the task action generation model can be a hybrid expert network, where sub-networks can be experts, each sub-network corresponds to one or more task types, and the task types corresponding to a sub-network are highly correlated.

[0062] For example, the first sub-network corresponds to refined operational tasks, such as threading a needle, surgical procedures, and precision assembly; the second sub-network corresponds to safety tasks, such as picking up fragile ceramic cups and handling flexible materials; and the third sub-network corresponds to efficiency tasks, such as high-speed sorting, throwing scattered toys into storage boxes, and neatly folding dried clothes. The task types corresponding to the sub-networks can also be customized according to the scenario or needs, and the number of sub-networks is not limited in this publication.

[0063] For example, a hybrid expert network may include multiple experts, each of whom focuses on learning one or more highly relevant sub-skills.

[0064] In some embodiments, the task action generation model may be pre-trained, which may be obtained by training each sub-network separately using different training data to obtain a task action generation model that integrates multiple sub-networks. Each sub-network may be a diffusion model or other neural network models, which are not limited in this disclosure.

[0065] In some embodiments, the task action generation model can be obtained by the execution subject of the task action generation method through model training, or it can be trained by other subjects and then sent to the execution subject of the task action generation method for model inference. This disclosure does not limit this.

[0066] Step 102: Based on the degree of matching between the task data and the sub-network, determine the target sub-network in at least one sub-network of the task action generation model.

[0067] In some embodiments, task data includes at least one of sensor images, joint positions, and voice commands.

[0068] In some embodiments, task data may be data obtained by feature extraction from sensor images, joint positions, language commands, etc.

[0069] For example, a shared observation encoder is initialized to extract feature representations from raw sensor data such as images and joint states.

[0070] In some embodiments, task data may be sensor images detected by sensors, such as real-time images captured by the robot's camera, which are used by the robot to mimic objects in the captured real-time images to perform corresponding tasks.

[0071] For example, an operator sorts items within the robot's detection range. The robot's camera captures images of the sorting action and controls the robot to follow the sorting action to sort the items.

[0072] In some embodiments, task data may be joint positions obtained through motion capture suits. The motion capture suits detect the movement positions of each joint of the operator in real time, so that the robot can follow the joint positions of the operator to perform the corresponding tasks.

[0073] For example, an operator wearing a motion capture suit is performing a surgical procedure. The robot can be controlled to perform the same operation as the operator based on the position of each joint detected by the motion capture suit.

[0074] In some embodiments, task data may be language instructions via voice prompts or screen input, which enable the robot to perform corresponding tasks according to the language instructions.

[0075] For example, if an operator gives a voice instruction to "fold the clothes on the table," the task action generation model can use this language instruction to control the robot to fold the clothes on the table.

[0076] In some embodiments, a target subnetwork is determined in at least one subnetwork of the task action generation model based on the degree of matching between the task data and the subnetwork. This can be achieved by using a pre-trained classification model to determine the subnetwork corresponding to the task data.

[0077] In some embodiments, the pre-trained classification model can be obtained by training an initial model using training data. The trained model can directly output the corresponding sub-network identifier based on the input task data.

[0078] In some embodiments, the target subnetwork is determined in at least one subnetwork based on the degree of matching between the task data and the subnetwork. This can be based on the task type corresponding to the task data, selecting subnetworks of the same type from multiple subnetworks. The same type can indicate a high degree of matching.

[0079] In some embodiments, the target subnetwork is determined in at least one subnetwork based on the degree of matching between the task data and the subnetwork. This can be done by clustering, calculating the Euclidean distance / distance between the task data and the center of each cluster, and determining the subnetwork corresponding to the cluster with the shortest distance as the target subnetwork. The shortest distance indicates a higher degree of matching.

[0080] In some embodiments, a target sub-network is determined based on task data. The task data can be assigned to the sub-network with the closest processing task type. This sub-network is trained using training data with the same or similar task data type, thereby ensuring that the most suitable sub-network is assigned to the task data, achieving accurate, stable and efficient multi-task operation.

[0081] Furthermore, the target subnetworks determined for different task data can be the same or different. Specifically, for task data with the same or similar task types, the determined target subnetworks are the same; for task data with different task types, the determined target subnetworks are likely to be different subnetworks.

[0082] In some embodiments, for a given task data, the determined target subnetwork may be one or more, and this disclosure does not limit this.

[0083] For example, if there is only one target subnetwork, the output of that target subnetwork can be used as the result; if there are multiple target subnetworks, the results of each target subnetwork's output can be weighted and calculated to determine the final result.

[0084] Step 103: Determine the target task action corresponding to the task data through the target sub-network.

[0085] In some embodiments, the target task action is used for the robot to perform the target task corresponding to the task data.

[0086] In some embodiments, determining the target task action corresponding to the task data through the target sub-network can be achieved by determining the network layer parameters of each network layer in the target sub-network based on the output task data, thereby obtaining the output target task action.

[0087] In some embodiments, the target task action can be the output of the network layer parameters corresponding to the minimum loss function achieved by the target subnetwork through multiple iterations.

[0088] In the above embodiments, by determining a target subnetwork in at least one subnetwork based on task data, and generating target task actions corresponding to the task data through the target subnetwork, the most suitable subnetwork is allocated based on the task data. This allows only the target subnetwork to be activated when using it, thereby reducing the overall computational cost of the model and improving the efficiency of task action generation. Furthermore, since each subnetwork is trained for a specific task type, the target task action is generated using the target subnetwork corresponding to the task data, which improves the accuracy of task action generation.

[0089] Figure 2 This is a flowchart illustrating another task action generation method proposed in this disclosure, based on... Figure 1 The embodiment shown, Figure 2 right Figure 1 Step 102 in the text will be further explained, such as Figure 2 As shown, the method includes the following steps:

[0090] Step 201: For the first sub-network in the task action generation model, determine the energy value corresponding to the task data and the first sub-network through a gating network.

[0091] In some embodiments, the energy value reflects the degree of matching between the task data and the first sub-network.

[0092] In some embodiments, the gated network may be a gated network incorporating an energy model, which can calculate an energy value between the task data and each subnetwork based on the task data, the energy value reflecting the degree of correlation between the task data and the subnetwork.

[0093] In some embodiments, each subnetwork in the gating network corresponds to an energy function, which is used to evaluate the degree of matching between the currently input task data and the subnetwork, i.e., the energy value. The lower the energy value, the higher the degree of matching between the task data and the subnetwork. In other words, the lower the energy value, the closer the task type of the task data is to the task type corresponding to the subnetwork.

[0094] For example, a gated network can be an EBM-based gated network that automatically analyzes the underlying patterns and internal relationships of all work orders (observation data) through an energy model, automatically forms clusters, and assigns tasks based on the matching degree between the work order and each cluster.

[0095] In some embodiments, the first sub-network can be any sub-network in the task action generation model. Through a gating network, the energy value corresponding to the task data and each first sub-network can be determined.

[0096] For example, for observations at time t Through gating network Generate energy values ​​for all experts Its parameters are The network is based on observation and experts As input, output a scalar value for subsequent expert calculations. For observation The energy value reflects the "mismatch" between the observation and the expert's area of ​​expertise. That is, if the observation is very much in line with the expert's expertise, the lower the energy value, and vice versa.

[0097] Step 202: Based on the energy value, determine the probability value corresponding to the task data and the first sub-network.

[0098] In some embodiments, the probability value is negatively correlated with the energy value.

[0099] In some embodiments, the probability value is determined based on the energy value, which can be achieved by converting the energy into a probability distribution using a softmax function. Specifically, the higher the energy value, the lower the probability value, and vice versa.

[0100] For example, combining pre-computed normalization constants on the training data The observation distribution of each expert was obtained. where exp() is the natural exponential function For observation Through gating network Generate energy values ​​for all experts Its parameters are , It is a pre-calculated normalization constant on the training data.

[0101] Step 203: The first subnetwork whose probability value meets the preset condition is determined as the target subnetwork.

[0102] In some embodiments, the preset condition can be the highest probability value, that is, the first subnetwork with the highest probability value is determined as the target subnetwork. The first subnetwork with the highest probability value has the highest degree of matching with the task data; in other words, the task type corresponding to the task data matches the task type corresponding to the first subnetwork.

[0103] In some embodiments, the preset condition can be the top N probability values ​​sorted from largest to smallest, where N can be customized according to the scenario or requirements, and this disclosure does not limit this. Then, the N first subnetworks are determined as target subnetworks, and task actions are generated simultaneously through these N target subnetworks.

[0104] In some embodiments, the preset conditions can be customized according to the scenario or requirements, and this disclosure does not limit this.

[0105] For example, the selection of experts is calculated based on Bayes' theorem. probability And select the expert with the highest probability. As the currently active expert.

[0106] In the above embodiments, by combining a gated network with an energy model, the energy value of the task data and each sub-network can be calculated in real time based on the input task data. The probability of each sub-network matching the task data can be calculated based on the energy value. Thus, the most suitable target sub-network can be selected from multiple sub-networks for task action generation. Automatic allocation can be achieved without manual labeling. The target task action is generated through the target sub-network. The "divide and conquer" strategy improves practicality and scalability, reduces the overall computational cost of the model, and improves generation efficiency.

[0107] Figure 3 This is a flowchart illustrating another task action generation method proposed in an embodiment of this disclosure. Figure 3 based on Figures 1-2 The illustrated embodiment is for Figure 1 Step 103 in the text will be further explained, such as Figure 3 As shown, the method includes the following steps:

[0108] Step 301: Sample the initial noise action sequence in a standard Gaussian distribution.

[0109] In some embodiments, the initial noise action sequence may be obtained by sampling in a standard Gaussian distribution according to a preset sampling dimension.

[0110] In some embodiments, the action sequence may include actions in T consecutive time steps, i.e., the action sequence is a T×n dimensional sequence, where n can be determined according to the robot's action space dimension. For example, for a robotic arm with n degrees of freedom, the action in a single time step can be represented as an n-dimensional vector, where each element represents the angle, angular velocity, or torque of a joint. The initial noisy action sequence can be a T×n dimensional matrix sampled from a standard Gaussian distribution, where each element is independently and identically distributed from the standard Gaussian distribution. Here, T is a pre-set time step, the value of which can be customized according to the scenario or requirements, and this disclosure does not limit it.

[0111] In some embodiments, the initial noise action sequence has the same shape as the action sequence.

[0112] In some embodiments, the sampled initial noise motion sequence may be scaled and / or translated according to the actual range of motion of each joint of the robot, so that the initial noise motion sequence conforms to the actual range of motion of each joint of the robot, which is not limited in this disclosure.

[0113] Step 302: Based on the task data, perform iterative denoising on the initial noisy action sequence to determine the target task action.

[0114] In some embodiments, the target task action includes the joint trajectory of at least one joint of the robot.

[0115] In some embodiments, iterative denoising can be performed by generating the target task action through multi-step reverse denoising.

[0116] In some embodiments, iterative denoising processing of the initial noisy action sequence based on task data may involve inputting the task data and the initial noisy action sequence into the target sub-network, predicting the noise contained in the initial noisy action sequence, denoising the action sequence based on the predicted noise, and iterating in a loop to generate the target task action.

[0117] In some embodiments, the target task action is a T×n dimensional matrix, including the joint trajectories of n joints of the robot within the next T time steps.

[0118] In the above embodiments, by performing iterative denoising processing based on task data through the target sub-network, target task actions that conform to the robot's motion space dimension can be generated, thereby determining the actions of each joint of the robot and improving the refinement and accuracy of the robot's task action generation.

[0119] Figure 4 This is a flowchart illustrating another task action generation method proposed in an embodiment of this disclosure. Figure 4 based on Figures 1-3 The illustrated embodiment is for Figure 3 Step 302 in the text will be further explained, such as Figure 4 As shown, the method includes the following steps:

[0120] Step 401: Input the initial noise action sequence and task data into the denoising model of the target sub-network to determine the predicted noise corresponding to the initial noise action sequence.

[0121] In some embodiments, the denoising model can be a diffusion model, which can predict the predicted noise corresponding to the initial noise action sequence based on the initial noise action sequence and task data.

[0122] Step 402: Based on the predicted noise, determine the first noise action sequence through denoising sampling processing.

[0123] In some embodiments, based on the predicted noise, the noise reduction sampling process can generate the next step, a noise-reduced action sequence, based on the predicted noise. That is, the predicted noise is extracted based on the initial noise action sequence to generate a first noise-reduced noise action sequence.

[0124] In some embodiments, the denoising sampling process can be a diffusible sampling algorithm, such as DDPM or DDIM, used to compute the next noise-reduced action sequence using the predicted noise.

[0125] Specifically, the denoising sampling process is the same as the diffusion sampling method in related technologies, and this disclosure does not limit it.

[0126] Step 403: Perform iterative denoising on the first noisy action sequence until the number of iterations reaches the preset number of iterations, and determine the target task action.

[0127] In some embodiments, the first noisy action sequence is subjected to iterative denoising processing, that is, the first noisy action sequence and the task data are input into the denoising model again to obtain the noise corresponding to the predicted first noisy action sequence, and based on the noise, the next step and the noise-reduced action sequence are generated. This process is repeated multiple times until the number of iterations reaches the preset number of iterations. The final noise-reduced action sequence is the target task action.

[0128] In the above embodiments, through iterative denoising processing, accurate target task actions can be generated based on task data. By assigning task data to a matching target sub-network, the trajectory of each joint can be generated using the corresponding target sub-network, thereby realizing task action generation based on task type.

[0129] Figure 5 This is a flowchart illustrating another task action generation method proposed in an embodiment of this disclosure. Figure 5 based on Figures 1-4 The illustrated embodiments will be further explained as follows: Figure 5 As shown, the method also includes the following steps:

[0130] Step 501: Send the target task action to the robot's controller to control at least one joint of the robot to perform the target task according to the joint trajectory in the target task action.

[0131] In some embodiments, the joint trajectory of at least one joint in the target task action is sent to the robot's controller, thereby enabling the controller to drive each joint of the robot to execute the corresponding joint trajectory in order to perform the corresponding target task.

[0132] For example, the hybrid expert model sends the output joint trajectory to the underlying tracking controller of the robotic arm, which drives the joint motors to accurately execute the motion planned by the trajectory.

[0133] In the above embodiments, since different task data can be assigned to different sub-networks, each sub-network can generate corresponding target task actions based on the input task data. Thus, the robot can execute different target tasks by executing different joint trajectories, thereby avoiding interference between trajectory generation between tasks and improving the overall performance of the task action generation model in multi-task scenarios.

[0134] Figure 6 A flowchart illustrating another task action generation method proposed in this disclosure embodiment is shown below. Figure 6 based on Figures 1-5 The illustrated embodiments will be further explained as follows: Figure 6 As shown, the method also includes the following steps:

[0135] Step 601: Obtain the initial training dataset.

[0136] In some embodiments, the initial training dataset includes at least one set of training data pairs, each training data pair including initial task data and the initial task action corresponding to the initial task data.

[0137] In some embodiments, the initial training dataset may be a batch of initial training data pairs randomly sampled from the training dataset, and the initial training data pairs may be observation-action pairs. ,in These are observations from a sampling batch. It refers to the action in the sampling batch, that is, for an initial task data, there is a corresponding initial task action.

[0138] Step 602: Determine the energy value of the initial task data and the second sub-network in at least one sub-network through the gating network.

[0139] In some embodiments, the second subnetwork can be any one of at least one subnetwork.

[0140] In some embodiments, the gating network can be the gating network currently in the sub-network model training process. During the joint training of the sub-network and the gating network, the sub-network model training process generates initial task data and energy values ​​corresponding to each second sub-network based on the current gating network.

[0141] In some embodiments, determining the energy value corresponding to the initial task data and the second subnetwork in at least one subnetwork through a gating network can be as follows: Figure 2 The specific implementation of the embodiment shown in step 201 will not be repeated here.

[0142] Step 603: Update the buffer corresponding to the second sub-network based on the energy value.

[0143] In some embodiments, the buffer includes at least one set of initial training data pairs.

[0144] In some embodiments, updating the buffer corresponding to the second sub-network based on the energy value can be achieved by calculating the probability value corresponding to the initial task data based on the energy value, obtaining the probability distribution of the second sub-network for each initial task data, and storing the initial training data pairs corresponding to the top S initial task data points, sorted by probability value from largest to smallest, into the buffer corresponding to the second sub-network. Here, S is a pre-set update number, the specific value of which can be customized according to the scenario or requirements, and this disclosure does not limit this.

[0145] In some embodiments, for each sampled initial training dataset, the initial training data pairs that meet the conditions are stored in a buffer based on the energy value, thereby realizing real-time updates to the buffer corresponding to each sub-network.

[0146] For example, a batch of observation-action pairs is randomly sampled from the training dataset. They follow the natural distribution of the environment. For each expert Utilize the current gating network to calculate each observation in the batch Preference probability: Based on this probability distribution, from For each expert sampling Its most "preferred" observation and its corresponding The data is stored in the expert's dedicated training buffer. In the middle. Among them, For all experts generated through the gating network, targeting Energy value, softmax This is the activation function used to convert energy values ​​into a probability distribution.

[0147] Step 604: Update the network layer parameters of the second sub-network based on at least one set of initial training data pairs in the buffer.

[0148] In some embodiments, updating the network layer parameters of the second sub-network based on at least one set of initial training data pairs in the buffer can be achieved by training the initial model based on the initial training data pairs in the buffer to obtain the parameters of each network layer in the second sub-network.

[0149] In some embodiments, the corresponding model training method may be adopted according to the type of the initial model of the second sub-network, which is not limited in this disclosure.

[0150] In the above embodiments, by combining a gating network, training data pairs are dynamically determined in the real-time updated training dataset to update the buffer for model training of the sub-network, thereby improving the real-time performance of the training data for model training and thus improving the accuracy of the joint trajectories generated by the sub-network.

[0151] Figure 7 This is a flowchart illustrating another task action generation method proposed in an embodiment of this disclosure. Figure 7 based on Figures 1-6 The embodiment shown, Figure 7 right Figure 6 Step 604 in the text will be further explained, such as Figure 7 As shown, it includes the following steps:

[0152] Step 701: Determine at least one first training data pair in the buffer.

[0153] In some embodiments, the first training data pair includes first initial task data and first initial task actions corresponding to the first initial task data.

[0154] In some embodiments, determining at least one set of first training data pairs in the buffer can be achieved by randomly sampling a batch of first training data pairs from the buffer of the second sub-network.

[0155] In some embodiments, the number of randomly selected first training data pairs can be customized according to the scenario or requirements, and this disclosure does not limit this.

[0156] For example, for each expert From its buffer Randomly select a small batch of data .

[0157] Step 702: Based on the first initial task data, determine the predicted task action through the second sub-network.

[0158] In some embodiments, based on the first initial task data, task actions are generated through the current second sub-network to obtain predicted task actions.

[0159] In some embodiments, generating the predicted task action corresponding to the first initial task data through the second sub-network can be achieved by adding noise to the first initial task action to obtain a noisy action sequence, and then inputting the noisy action sequence and the first initial task data into the second sub-network. The second sub-network performs forward computation based on a denoising model of U-Net or Transformer architecture and outputs its predicted value for the added noise, i.e., the predicted task action.

[0160] In some embodiments, the network layer parameters of the current second sub-network can be the initial network layer parameters, which correspond to the current gated network.

[0161] Step 703: Determine the network layer parameters of the second sub-network based on the predicted task action and the first initial task action corresponding to the first initial task data.

[0162] In some embodiments, the network layer parameters of the second sub-network are determined based on the predicted task action and the first initial task action corresponding to the first initial task data. This can be achieved by calculating the difference between the predicted task action and the first initial task action, which is quantized using a loss function. Backpropagation is then performed on this loss to calculate its gradient with respect to all trainable network layer parameters of the second sub-network. A gradient descent optimization algorithm is then used to update the network layer parameters of the second sub-network based on the calculated gradient.

[0163] In some embodiments, the loss function can be the denoising loss of the diffusion model:

[0164] ,in, These are the neural network parameters of the diffusion model; π(o|e) is the observation distribution associated with the subnetwork e, used to characterize the expert's preference for a specific initial task data o; MSE is the mean squared error. It is the actual noise calculated based on the training data during the diffusion process; This is the noise predicted by the sub-network. In the above formula, i represents the i-th training sample, and k represents the k-th iteration in the denoising step.

[0165] In some embodiments, by iteratively executing steps 702 and 703, the network parameters of the second sub-network can be updated and optimized to minimize the action prediction error of the second sub-network.

[0166] In the above embodiments, by updating the buffer of the second sub-network in real time, the first initial training data pair for training can be randomly extracted from the real-time updated buffer to update the network layer parameters of the second sub-network, so that the updated second sub-network can accurately generate task actions.

[0167] Figure 8 This is a flowchart illustrating another task action generation method proposed in an embodiment of this disclosure. Figure 8 based on Figures 1-7 The illustrated embodiments will be further explained as follows: Figure 8 As shown, the method also includes the following steps:

[0168] Step 801: Input the initial task data from the initial training dataset into the gating network to determine the probability distribution corresponding to the initial task data.

[0169] In some embodiments, the initial training dataset is used to train the gated network, that is, the initial training data in the initial training dataset is used to update the parameters of the gated network.

[0170] In some embodiments, the initial task data from the initial training dataset is input into the current gating network, which can determine the probability distribution of the initial task data for each sub-network.

[0171] Step 802: Based on the optimized loss function, update the parameters of the gated network through gradient descent.

[0172] In some embodiments, the optimization loss function includes a reconstruction objective, a specialization objective, and a coverage objective.

[0173] In some embodiments, the optimization loss function can be:

[0174] The reconstruction target can be This is used to encourage subnetworks to make accurate action predictions based on their preferred task data; specialization goals can be... It is used to encourage gating networks to explicitly assign data for each task to specific subnetworks, reduce overlap between subnetworks, and promote skill separation. β This is the regularization coefficient, used to control the weights of information entropy or KL penalty. It refers to the distribution of experts in the observation space; coverage targets can be... This is used to encourage the probability distribution of each sub-network to be not too narrow, so as to ensure that all sub-networks can jointly cover the space of all task data and avoid any area being ignored by all sub-networks. This represents the energy value.

[0175] In some embodiments, the parameters of the gated network are updated by gradient descent based on the optimized loss function. This can be achieved by using a gradient descent optimization algorithm in related technologies, which calculates the gradient corresponding to all parameters of the gated network by calculating the optimized loss function, and then adjusts the parameters of the gated network according to the update rules.

[0176] In some embodiments, the loss function of the second sub-network and the loss function of the gated network can be combined and defined to form a joint optimization loss function:

[0177]

[0178] in, π(o|e) represents the expected distribution of observation o, which is determined by the selected expert e; π(o|e) is the observation distribution associated with e, used to characterize the degree of preference of the expert for a specific task data o; π(a|o,e) represents the expectation of the policy distribution for action a, i.e., the action distribution of the policy output given observation o and expert e; for This is used to measure the expert's learning level of the action (how well they have learned the action) on the task data o that e is currently good at. For the actual action, perform MSE with the action 'a' output by the expert to measure the expert's fitting error on the task data 'o'. Gated networks are encouraged to explicitly assign data for each task to a small number of experts, reducing overlap between experts and promoting skill separation. It refers to the distribution of experts within the observation space; β This is the regularization coefficient, used to control the weight of information entropy or KL penalty; This mathematical form is information entropy, which is the expected value of the negative logarithmic probability, where Let be the information entropy of the expert distribution. The joint optimization loss function described above encourages that the probability distribution π(o|e) of each expert is not too narrow, so as to ensure that all experts can jointly cover the space of all task data and avoid any region being ignored by all experts.

[0179] Based on this joint optimization loss function, the parameters of the second sub-network and the gated network are updated through gradient descent to achieve alternating optimization of the gated network and the sub-network.

[0180] In the above embodiments, by updating the parameters of the gating network in real time, the performance of the gating network can be improved, thereby increasing the accuracy of sub-network allocation for subsequent task data.

[0181] In summary, the task action generation method proposed in this disclosure, through a divide-and-conquer strategy, enables each sub-network to be responsible for generating task actions within its professional domain, thereby reducing the computational cost of the model, eliminating manual intervention, improving adaptability, meeting real-time requirements, and improving the efficiency and accuracy of task action generation.

[0182] The following is a specific implementation of a task action generation method provided in this disclosure:

[0183] Figure 9 This is an overall architecture diagram of a method for constructing a robotic arm behavior model based on a hybrid expert network and energy model. (See diagram for example.) Figure 9 As shown, the core of this method lies in the collaborative work of the encoder, the gating network, and multiple experts. That is, through an automated, data-driven skill decomposition and allocation mechanism, the model can learn and master a variety of basic skills, thereby performing well in complex multi-task scenarios.

[0184] Training phase: Observing input After passing through the encoder, the data is sent to the gating network. The gating network calculates the probability distribution of the observation belonging to each expert. Based on this, the data is dynamically allocated to different experts for training. Each expert independently learns strategies in their area of ​​expertise.

[0185] Inference phase: Observing input Similarly, after passing through the encoder and gating network, the gating network selects the expert with the highest probability (argmax π(o|e)) and activates only that expert to generate the final control command. .

[0186] Includes the following steps:

[0187] Step 1: Model Initialization

[0188] Initialize the following key components:

[0189] A shared observation encoder is used to extract feature representations from raw sensor data such as images and joint states.

[0190] A gated network Its parameters are The network uses observation coding and expert indexing. As input, output a scalar value for subsequent expert calculations. For observation Preferred energy.

[0191] K expert networks Each expert It is a behavioral model (such as a diffusion model) responsible for the behavior of a given observation. Predicting the movements of the robotic arm under the given conditions Each expert corresponds to a specific basic skill.

[0192] Step Two: Joint Training Process

[0193] This step aims to make each expert more specialized on their preferred subset of data. The model is trained by alternately optimizing the gating network and the expert network, with the goal of minimizing the overall loss function across all training data. :

[0194] in, π(o|e) represents the expected distribution of observation o, which is determined by the selected expert e; π(o|e) is the distribution of observations associated with the selected expert e, used to characterize the degree of preference of the expert for a specific observation o. π(a|o,e) represents the expectation of the policy distribution for action a, i.e., the action distribution of the policy output given observation o and expert e; for This is used to measure how well the subnetwork e learns the actions on the observation o that it is currently good at. For the actual action, perform MSE with the action 'a' output by the expert to measure the expert's fitting error on the task data 'o'. Gated networks are encouraged to explicitly assign each observation to a small number of experts, reducing overlap between subnetworks and promoting skill separation. It refers to the distribution of experts within the observation space; This mathematical form is information entropy, which is the expected value of the negative logarithmic probability. Let be the information entropy of the expert distribution; β is the regularization coefficient, used to control the weight of the information entropy or KL penalty. This encourages that the probability distribution π(o|e) of each expert is not too narrow, ensuring that all experts can collectively cover the entire observation space and avoiding any regions being ignored by all experts.

[0195] The specific iterative process includes:

[0196] 1. Data Sampling: Randomly sample a batch of observation-action pairs from the training dataset. They follow the natural distribution of the environment. .

[0197] 2. Expert allocation and buffer update:

[0198] For each expert Utilize the current gating network to calculate each observation in the batch Preference probability:

[0199]

[0200] Based on this probability distribution, from For each expert sampling Its most "preferred" observation and its corresponding The data is stored in the expert's dedicated training buffer. middle.

[0201] 3. Expert strategy update:

[0202] For each expert From its buffer Randomly select a small batch of data .

[0203] Based on this data, by minimizing the experts The action prediction error (such as the denoising loss of the diffusion model) is used to update the parameters of the expert network:

[0204] in, These are the neural network parameters of the diffusion model; π(o|e) is the observation distribution associated with the subnetwork e, used to characterize the expert's preference for a specific initial task data o; MSE is the mean squared error. It is the actual noise calculated based on the training data during the diffusion process; This is the noise predicted by the sub-network. In the above formula, i represents the i-th training sample, and k represents the k-th iteration in the denoising step.

[0205] 4. Observe the distribution update:

[0206] Use the complete batch sampled in step 1 Update the gated network parameters :

[0207]

[0208] Its optimization objective consists of three parts:

[0209] (1) Reconstruction loss: This encourages experts to make accurate predictions about actions based on their observations of preferences.

[0210] (2) Specialization incentives: This approach encourages gating networks to explicitly assign each observation to a specific expert, reducing overlap between experts and promoting skill separation. It represents the distribution of experts in the observation space, where β is the regularization coefficient used to control the weights of information entropy or KL penalty.

[0211] (3) Entropy regularization term: This encourages the distribution of observations from each expert. The scope should not be too narrow, so as to ensure that all experts can jointly cover the entire observation space and avoid some areas being overlooked by all experts.

[0212] Step 3: Real-time Reasoning and Decision Making

[0213] When the model is deployed on a real robotic arm for operation, for each new observation at each time step Perform the following procedure:

[0214] 1. Expert selection:

[0215] calculate For the energy values ​​of all experts .

[0216] Combined with pre-computed normalization constants on the training data The observation distribution of each expert was obtained. Where exp() is the natural exponential function For observation Through gating network Generate energy values ​​for all experts Its parameters are .

[0217] Experts are selected based on Bayes' theorem. probability And select the expert with the highest probability. As the currently activated expert. (This sparse activation mechanism with k=1 is crucial; it ensures efficiency during inference and meets real-time control requirements.)

[0218] Action generation: Observation and expert index Input into the corresponding expert network In the process, the expert generates the final control actions. And send it to the robotic arm for execution.

[0219] In summary, the beneficial effects of the above scheme are as follows:

[0220] (1) Improve multi-task learning performance and overcome the average effect.

[0221] (2) Fully automated skill decomposition.

[0222] (3) Enhance the model’s ability to model complex, multimodal action distributions.

[0223] Figure 10 This is a schematic diagram of the structure of a task action generation device 1000 according to a disclosed embodiment. Figure 10 As shown, the device includes: an acquisition module 1010, an allocation module 1020, and a generation module 1030.

[0224] The acquisition module is used to acquire the task action generation model, which includes at least one sub-network, each of which is trained to correspond to at least one task type.

[0225] The allocation module is used to determine the target subnetwork in at least one subnetwork of the task action generation model based on the degree of matching between task data and subnetworks. The task data includes at least one of sensor images, joint positions, and language commands.

[0226] The generation module is used to determine the target task action corresponding to the task data through the target sub-network. The target task action is used by the robot to execute the target task corresponding to the task data.

[0227] In some embodiments, the allocation module is further configured to, for the first sub-network in the task action generation model, determine the energy value corresponding to the task data and the first sub-network through a gating network, wherein the energy value reflects the degree of matching between the task data and the first sub-network; determine the probability value corresponding to the task data and the first sub-network based on the energy value, wherein the probability value is negatively correlated with the energy value; and determine the first sub-network whose probability value meets the preset condition as the target sub-network.

[0228] In some embodiments, the generation module is further configured to sample an initial noisy action sequence in a standard Gaussian distribution; based on the task data, perform iterative denoising processing on the initial noisy action sequence to determine the target task action, the target task action including the joint trajectory of at least one joint of the robot.

[0229] In some embodiments, the generation module is further configured to input the initial noise action sequence and task data into the denoising model of the target sub-network, determine the predicted noise corresponding to the initial noise action sequence; determine the first noise action sequence based on the predicted noise through denoising sampling processing; perform iterative denoising processing on the first noise action sequence until the number of iterations reaches the preset number of iterations, and determine the target task action.

[0230] In some embodiments, the device further includes a transceiver module for sending the target task action to the robot's controller to control at least one joint of the robot to perform the target task according to the joint trajectory in the target task action.

[0231] In some embodiments, the apparatus further includes a training module for acquiring an initial training dataset, the initial training dataset including at least one set of training data pairs, each training data pair including initial task data and an initial task action corresponding to the initial task data; determining the energy value corresponding to the initial task data and a second sub-network in at least one sub-network through a gating network; updating the buffer corresponding to the second sub-network based on the energy value, the buffer including at least one set of initial training data pairs; and updating the network layer parameters of the second sub-network based on the at least one set of initial training data pairs in the buffer.

[0232] In some embodiments, the training module is further configured to determine at least one first training data pair in the buffer, the first training data pair including first initial task data and a first initial task action corresponding to the first initial task data; determine a predicted task action based on the first initial task data through a second sub-network; and determine the network layer parameters of the second sub-network based on the predicted task action and the first initial task action corresponding to the first initial task data.

[0233] In some embodiments, the training module is further configured to input initial task data from the initial training dataset into the gating network, determine the probability distribution corresponding to the initial task data, and update the parameters of the gating network through gradient descent based on the optimized loss function. The optimized loss function includes a reconstruction objective, a specialization objective, and a coverage objective.

[0234] The task action generation device proposed in this disclosure uses a gating network to allocate each input task data to a target sub-network in the task action generation model, thereby activating the target sub-network to generate task actions. This avoids interference between tasks, achieves decoupling between tasks, reduces the computational cost of the model, improves the efficiency and accuracy of task action generation, and further enhances the system's adaptability and cross-task scalability.

[0235] This disclosure also proposes a robot configured to perform the above-described task action generation method, or including the above-described task action generation device.

[0236] Figure 11 This is a schematic diagram of the structure of an electronic device 1100 for implementing the above-described task action generation method, according to an exemplary embodiment.

[0237] Reference Figure 11 The electronic device 1100 may include one or more of the following components: a processing component 1102, a memory 1104, a power supply component 1106, a multimedia component 1108, an audio component 1110, an input / output (I / O) interface 1112, a sensor component 1114, and a communication component 1116.

[0238] Processing component 1102 typically controls the overall operation of electronic device 1100, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 1102 may include one or more processors 1120 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 1102 may include one or more modules to facilitate interaction between processing component 1102 and other components. For example, processing component 1102 may include a multimedia module to facilitate interaction between multimedia component 1108 and processing component 1102.

[0239] Memory 1104 is configured to store various types of data to support the operation of electronic device 1100. Examples of such data include instructions for any application or method operating on electronic device 1100, contact data, phonebook data, messages, pictures, videos, etc. Memory 1104 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0240] Power supply component 1106 provides power to various components of electronic device 1100. Power supply component 1106 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 1100.

[0241] Multimedia component 1108 includes a screen that provides an output interface between electronic device 1100 and a user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 1108 includes a front-facing camera and / or a rear-facing camera. When electronic device 1100 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0242] Audio component 1110 is configured to output and / or input audio signals. For example, audio component 1110 includes a microphone (MIC) configured to receive external audio signals when electronic device 1100 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 1104 or transmitted via communication component 1116. In some embodiments, audio component 1110 also includes a speaker for outputting audio signals.

[0243] I / O interface 1112 provides an interface between processing component 1102 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0244] Sensor assembly 1114 includes one or more sensors for providing state assessments of various aspects of electronic device 1100. For example, sensor assembly 1114 may detect the on / off state of electronic device 1100, the relative positioning of components such as the display and keypad of electronic device 1100, changes in position of electronic device 1100 or a component of electronic device 1100, the presence or absence of user contact with electronic device 1100, the orientation or acceleration / deceleration of electronic device 1100, and temperature changes of electronic device 1100. Sensor assembly 1114 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 1114 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 1114 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0245] Communication component 1116 is configured to facilitate wired or wireless communication between electronic device 1100 and other devices. Electronic device 1100 can access wireless networks based on communication standards, such as WiFi, 2G or 3G, 4G LTE, 5G NR (NewRadio), or combinations thereof. In one exemplary embodiment, communication component 1116 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 1116 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0246] In an exemplary embodiment, the electronic device 1100 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0247] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1104 including instructions, which can be executed by a processor 1120 of an electronic device 1100 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0248] Embodiments of this disclosure also propose a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to perform the task action generation method described in the above embodiments of this disclosure.

[0249] Embodiments of this disclosure also provide a computer program product, including a computer program that is executed by a processor using the task action generation method described in the above embodiments of this disclosure.

[0250] Embodiments of this disclosure also provide a chip including at least one processor and a communication interface; the communication interface is used to receive signals input to the chip or signals output from the chip, and the processor communicates with the communication interface and implements the method described in any one of the first aspects of this disclosure through logic circuits or executing code instructions.

[0251] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0252] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0253] It should be understood that various parts of the embodiments of this disclosure can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0254] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into a single processing module, or each unit can exist physically separately, or two or more units can be integrated into a single module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The aforementioned storage medium can be a read-only memory, a hard disk, or an optical disk, etc.

[0255] Although embodiments of the present disclosure have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present disclosure.

Claims

1. A task action generation method characterized by comprising: The method includes: Obtain a task action generation model, the task action generation model including at least one sub-network, each sub-network being trained to correspond to at least one task type; Based on the degree of matching between task data and sub-networks, a target sub-network is determined in at least one sub-network of the task action generation model. The task data includes at least one of sensor images, joint positions, and language commands. The degree of matching is determined by the energy value of the task data and the sub-network calculated based on the gating network. The target sub-network is used to determine the target task action corresponding to the task data, and the target task action is used by the robot to perform the target task corresponding to the task data.

2. The method of claim 1, wherein, The step of determining the target subnetwork in at least one subnetwork of the task action generation model based on the degree of matching between task data and subnetworks includes: For the first sub-network in the task action generation model, the energy value corresponding to the task data and the first sub-network is determined through the gating network. The energy value reflects the degree of matching between the task data and the first sub-network. Based on the energy value, a probability value corresponding to the task data and the first sub-network is determined, and the probability value is negatively correlated with the energy value; The first subnetwork whose probability value meets the preset condition is determined as the target subnetwork.

3. The method according to claim 1 or 2, characterized in that, Determining the target task action corresponding to the task data through the target sub-network includes: Sample the initial noisy action sequence in a standard Gaussian distribution; Based on the task data, the initial noisy action sequence is subjected to iterative denoising processing to determine the target task action, which includes the joint trajectory of at least one joint of the robot.

4. The method according to claim 3, characterized in that, The iterative denoising process for the initial noisy action sequence based on the task data includes: The initial noise action sequence and the task data are input into the denoising model of the target sub-network to determine the predicted noise corresponding to the initial noise action sequence; Based on the predicted noise, a first noise action sequence is determined through denoising sampling processing; The first noisy action sequence is subjected to the iterative denoising process until the number of iterations reaches the preset number of iterations, and the target task action is determined.

5. The method according to claim 1, characterized in that, The method further includes: The target task action is sent to the robot's controller to control at least one joint of the robot to perform the target task according to the joint trajectory in the target task action.

6. The method according to claim 1, characterized in that, The method further includes: Obtain an initial training dataset, which includes at least one set of training data pairs, each training data pair including initial task data and the initial task action corresponding to the initial task data; The energy value corresponding to the second sub-network in the at least one sub-network is determined by the gating network; Based on the energy value, update the buffer corresponding to the second sub-network, the buffer including at least one set of initial training data pairs; The network layer parameters of the second sub-network are updated based on at least one set of initial training data pairs in the buffer.

7. The method according to claim 6, characterized in that, The step of updating the network layer parameters of the second sub-network based on at least one set of initial training data pairs in the buffer includes: At least one first training data pair is determined in the buffer, the first training data pair including first initial task data and a first initial task action corresponding to the first initial task data; Based on the first initial task data, the predicted task action is determined through the second sub-network; Based on the predicted task action and the first initial task action corresponding to the first initial task data, the network layer parameters of the second sub-network are determined.

8. The method according to claim 6, characterized in that, The method further includes: The initial task data in the initial training dataset is input into the gating network to determine the probability distribution corresponding to the initial task data. The parameters of the gated network are updated by gradient descent based on the optimized loss function, which includes a reconstruction objective, a specialization objective, and a coverage objective.

9. A task action generation device, characterized in that, The task action generation device is configured to perform the method according to any one of claims 1-8.

10. A robot, characterized in that, It is configured to perform the method of any one of claims 1-8, or to include the task action generation apparatus of claim 9.

11. An electronic device, characterized in that, include: A processor and a memory for storing a computer program capable of running on the processor, wherein the processor, when running the computer program, performs the method of any one of claims 1 to 8.

12. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 8.

13. A program product, characterized in that, Includes computer instructions for causing a computer to perform the method of any one of claims 1 to 8.

14. A chip, characterized in that, It includes at least one processor and a communication interface; the communication interface is used to receive signals input to the chip or signals output from the chip, and the processor communicates with the communication interface and implements the method as described in any one of claims 1-8 through logic circuits or executing code instructions.

Images