Task execution method and device based on skill modularization, robot and medium

Abstract

This invention relates to the field of intelligent decision-making technology, and discloses a task execution method, device, robot, and medium based on skill modularization. The method includes: acquiring the robot's historical execution trajectory and segmenting and clustering it to obtain multiple atomic skill categories; performing parametric modeling on each atomic skill category to obtain a skill template; encoding and encapsulating the skill template to generate atomic skill modules, and storing them in a skill library; receiving a task description and parsing it into a sequence of sub-targets; performing a combined search of the atomic skill modules in the skill library based on the sub-target sequence, preconditions, and post-effects to generate a target skill chain; performing temporal coordination and parameter binding on the skill chain to generate an executable instruction sequence, and controlling the robot to execute the executable instruction sequence. This invention can be applied to robot task execution scenarios in the medical and fintech fields, enabling efficient reuse of skill modules and improving the flexibility and adaptability of robot task execution.

Description

Technical Field

[0001] This application relates to the field of intelligent decision-making technology, which can be applied to the medical and financial technology fields, and in particular to a task execution method, device, robot and medium based on skill modularization. Background Technology

[0002] With changing social structures and accelerated digitalization, the demand for elderly care robots in the healthcare field has increased significantly. These robots need to handle highly personalized care tasks such as item delivery, medication classification and management, fall detection and response, and assisted rehabilitation training. These tasks often present complex combinations due to dynamic changes in the user's condition. For example, in medication delivery scenarios, the robot must accurately identify the type of medication, plan its movement path, and execute a safe delivery. Meanwhile, in the financial services sector, branch service robots face diverse business needs such as identity verification, document printing, cash handling, and financial advisory guidance. The randomness of customer instructions leads to highly uncertain and dynamically combined task content, making it difficult to cover all possible situations with fixed processes. The core challenge in both scenarios lies in the personalization of user instructions and the inexhaustible nature of task processes, requiring robots to flexibly adapt to new task combinations.

[0003] Current robotic task execution primarily relies on end-to-end learning or hard-coded programming paradigms. For each specific task, developers need to re-collect large amounts of demonstration data to train models or write complete scripts. This repetitive development model not only consumes significant human resources but also makes the task adaptation process rigid, making it difficult to respond to dynamically changing user needs. Therefore, the fundamental flaw in existing technologies stems from the lack of a systematic skill modularization mechanism. Robots cannot encapsulate their mastered operational capabilities into standardized, reusable skill units, nor can they dynamically combine these units when faced with unseen tasks. Skills are isolated from each other, failing to form a structured knowledge base, resulting in a lack of flexibility in handling task combinations. For example, when a new task with multiple sub-objectives needs to be executed, the system cannot automatically retrieve and assemble existing skill modules based on preconditions and post-effects; the entire execution process must be redeveloped. This directly leads to low skill reuse rates and high deployment costs for new tasks, severely restricting the practical application of robots in dynamic environments such as elderly care, healthcare, and financial services. Summary of the Invention

[0004] The purpose of this application is to propose a task execution method, device, robot, and medium based on skill modularization, which can efficiently reuse skill modules, reduce the deployment cost of new tasks, and improve the flexibility and adaptability of robot task execution.

[0005] To address the aforementioned technical problems, embodiments of this application provide a task execution method based on skill modularization, comprising: The robot's historical execution trajectory is obtained, and the historical execution trajectory is segmented and clustered to obtain multiple atomic skill categories; Parametric modeling is performed on each atomic skill category to obtain a skill template with a parameter interface; The skill template is encoded and encapsulated to generate atomic skill modules, and the atomic skill modules are stored in the skill library. Each atomic skill module records the preconditions and post-effects. Receive a task description and parse the task description into a sequence of sub-targets; In the skill library, the atomic skill modules are combined and searched according to the sub-target sequence, the preconditions, and the post-effects to generate a target skill chain; The skill chain is time-coordinated and parameter-bound to generate an executable instruction sequence, and the robot is controlled to execute the executable instruction sequence.

[0006] To address the aforementioned technical problems, embodiments of this application provide a task execution device based on skill modularization, comprising: The historical execution trajectory acquisition module is used to acquire the robot's historical execution trajectory, and to segment and cluster the historical execution trajectory to obtain multiple atomic skill categories; The parametric modeling module is used to perform parametric modeling on each atomic skill category to obtain skill templates with parameter interfaces; A skill template encapsulation module is used to encode and encapsulate the skill template to generate atomic skill modules and store the atomic skill modules in the skill library. Each atomic skill module records the preconditions and post-effects. The task description parsing module is used to receive a task description and parse the task description into a sub-target sequence. The target skill chain generation module is used to perform a combined search on the atomic skill modules in the skill library based on the sub-target sequence, the preconditions, and the post-effects to generate a target skill chain; The instruction sequence execution module is used to perform timing coordination and parameter binding on the skill chain, generate an executable instruction sequence, and control the robot to execute the executable instruction sequence.

[0007] To solve the above-mentioned technical problems, one technical solution adopted by the present invention is to provide a robot, including one or more processors; and a memory for storing one or more programs, so that the one or more processors implement the task execution method based on skill modularization as described above.

[0008] To solve the above-mentioned technical problems, one technical solution adopted by the present invention is: a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the task execution method based on skill modularization described above.

[0009] This invention provides a task execution method, apparatus, robot, and medium based on skill modularization. The method includes: acquiring the robot's historical execution trajectory, segmenting and clustering the historical execution trajectory to obtain multiple atomic skill categories; performing parametric modeling on each atomic skill category to obtain a skill template with a parameter interface; encoding and encapsulating the skill template to generate atomic skill modules, and storing the atomic skill modules in a skill library, wherein each atomic skill module records preconditions and post-effects; receiving a task description and parsing the task description into a sequence of sub-targets; performing a combined search of atomic skill modules in the skill library based on the sub-target sequences, preconditions, and post-effects to generate a target skill chain; performing temporal coordination and parameter binding on the skill chain to generate an executable instruction sequence, and controlling the robot to execute the executable instruction sequence. This invention, by modularizing and dynamically combining skills, achieves rapid adaptation to new tasks, efficiently reusing skill modules, reducing the deployment cost of new tasks, and improving the flexibility and adaptability of robot task execution. Attached Figure Description

[0010] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram of an application environment for a task execution method based on skill modularization according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating the implementation of the task execution method based on skill modularization provided in this application embodiment; Figure 3 yes Figure 2 A flowchart illustrating a specific implementation method of step S1; Figure 4 yes Figure 2 A flowchart illustrating a specific implementation method of step S2; Figure 5 yes Figure 2 A flowchart illustrating a specific implementation method of step S3; Figure 6 yes Figure 2A flowchart illustrating a specific implementation of step S5; Figure 7 yes Figure 2 A flowchart illustrating a specific implementation of step S6; Figure 8 yes Figure 2 A flowchart illustrating a specific implementation method following step S6; Figure 9 This is a schematic diagram of a task execution device based on skill modularization provided in an embodiment of this application; Figure 10 This is a schematic diagram of the robot provided in an embodiment of this application. Detailed Implementation

[0012] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0013] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0014] Traditional robotic task execution methods primarily employ end-to-end learning or programming paradigms. For each specific task, developers need to collect extensive demonstration data to retrain the model or write fixed process scripts covering all steps. This paradigm lacks a systematic framework for modularizing and combining skills, preventing robots from encapsulating learned operational capabilities into reusable standardized units, and hindering their ability to dynamically assemble existing skills into new behavioral sequences when faced with unseen task combinations. In summary, existing technologies suffer from the problems of modular reuse and dynamic combination of robotic skills, and high costs associated with adapting to new tasks.

[0015] To address this, this application proposes a task execution method based on skill modularization, comprising: acquiring the robot's historical execution trajectory, segmenting and clustering the historical execution trajectory to obtain multiple atomic skill categories; performing parameterized modeling on each atomic skill category to obtain a skill template with parameter interfaces; encoding and encapsulating the skill template to generate atomic skill modules, and storing the atomic skill modules in a skill library, with each atomic skill module recording preconditions and post-effects; receiving a task description and parsing the task description into a sequence of sub-targets; in the skill library, performing a combined search on the atomic skill modules based on the sub-target sequences, preconditions, and post-effects to generate a target skill chain; performing temporal coordination and parameter binding on the skill chain to generate an executable instruction sequence, and controlling the robot to execute the executable instruction sequence.

[0016] For ease of understanding, the following explains some key terms in this embodiment: Historical execution trajectory: refers to the sequence of motion data generated during the actual operation of the robot, such as joint angles and end effector poses, which record the detailed process of the robot completing a specific action or task.

[0017] Atomic skill categories: These refer to sets of skills representing the basic operational units of a robot, obtained by segmenting and clustering historical execution trajectories. Each category aggregates trajectory fragments with similar motion characteristics and semantics.

[0018] Skill template: refers to a general motion pattern with adjustable parameter interfaces, formed after parametric modeling of atomic skill categories. It can abstract the commonalities of a certain type of skill and adapt to different specific execution scenarios through parameter interfaces.

[0019] Atomic skill modules: These are independent, reusable units that encapsulate skill templates, parameter interfaces, prerequisites, and post-effects. Each module represents a basic robot operation that can be executed independently.

[0020] Skill Repository: Refers to a knowledge base used to store all encapsulated atomic skill modules. It provides a unified platform for skill retrieval, combination, and management.

[0021] Task description: This refers to the natural language or structured description entered by the user that the robot expects to complete. It is the initial input for the robot to understand the user's intention.

[0022] Sub-goal sequence: refers to a series of intermediate goals that the robot needs to achieve sequentially, obtained after parsing the task description. It decomposes the high-level abstract task into actionable steps.

[0023] Target skill chain: refers to the sequence of skills to be executed, obtained by searching and combining skills from a skill library based on sub-target sequences and the pre- and post-conditions of skill modules, for completing a specific task. It is the specific planning for a robot to perform a task.

[0024] Executable instruction sequence: refers to the low-level control instructions that a robot can directly understand and execute after the target skill chain has been time-coordinated and parameter-bound. It drives the robot's actual actions.

[0025] This application constructs a reusable skill library by automatically extracting, parametrically modeling, and encapsulating atomic skill modules from the robot's historical execution trajectories. It then dynamically combines and searches for target skill chains based on task descriptions, followed by temporal coordination and parameter binding to generate an executable instruction sequence. This solves the problems of modular reuse and dynamic combination of skills, and high costs associated with adapting to new tasks, in traditional robot task execution methods. It significantly improves the adaptability and development efficiency of robots in personalized and uncertain task scenarios such as elderly care and financial services.

[0026] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0027] The action generation method based on multimodal pre-training provided in this invention can be applied to, for example... Figure 1 In this application environment, the user communicates with the robot via a network. The robot can receive the user's task description, generate an executable instruction sequence based on the task description, and execute the executable instruction sequence. The invention will now be described in detail through specific embodiments.

[0028] The skill-modular task execution method provided in this application can be applied to robot task execution scenarios in the medical and fintech fields.

[0029] Please see Figure 2 , Figure 2 A specific implementation of a task execution method based on skill modularization is shown.

[0030] It should be noted that if substantially the same result is obtained, the method of this invention is not based on... Figure 2 Limited to the order of the processes shown, this method includes the following steps: S1: Obtain the robot's historical execution trajectory, and segment and cluster the historical execution trajectory to obtain multiple atomic skill categories.

[0031] Specifically, historical execution trajectories can be recorded in real time by the robot's internal sensor system, such as encoder data for each joint and position and posture data of the end effector. These continuous trajectory data can be segmented using methods based on fixed time windows or preset kinematic thresholds (e.g., velocity or acceleration rate of change exceeding a specific value). The resulting trajectory segments can be clustered using clustering algorithms such as K-means or DBSCAN based on their original kinematic characteristics (e.g., joint angle changes, end effector pose changes) to group similar trajectory segments into the same atomic skill category. For example, when a robot performs a "grasp" action multiple times, its trajectory data from the initial movement to contacting the object, gripping the object, and lifting the object is recorded. This data is segmented into multiple smaller segments, such as "moving above the object," "descending to contact the object," "closing the gripper," and "lifting the object," and then similar segments are grouped together to form the atomic skill category of "grasp."

[0032] Please see Figure 3 , Figure 3 A specific implementation of step S1 is shown below: S11: Obtain the robot's historical execution trajectory, wherein the historical execution trajectory includes a joint angle sequence and an end effector pose sequence. S12: Identify skill boundary points in the historical execution trajectory. S13: Segment the historical execution trajectory based on the skill boundary points to obtain multiple trajectory segments. S14: Extract kinematic and semantic features from each trajectory segment. S15: Cluster the kinematic and semantic features to group similar trajectory segments into the same atomic skill category.

[0033] Specifically, when acquiring the robot's historical execution trajectory, the historical execution trajectory refers to the set of motion data recorded when the robot performed tasks in the past. This data is usually stored in the form of a time series, containing information about the complete motion process of the robot from the start to the end of the task. Its purpose is to provide original experience data for the learning and reuse of robot skills. The historical execution trajectory includes joint angle sequences and end effector pose sequences. Among them, the joint angle sequence describes the set of angle values ​​of each joint of the robot at different points in time. For example, for a multi-degree-of-freedom manipulator, its joint angle sequence can record the precise position of each rotational or translational joint at each sampling moment, which reflects the internal motion state of the robot body. This can be achieved by directly reading the encoder data of each joint through the robot controller, or by deriving it from the end effector pose through inverse kinematics. The end effector pose sequence describes the sequence of changes in the position and orientation of the robot's end effector in three-dimensional space over time. The position is usually represented by X, Y, and Z coordinates in the Cartesian coordinate system, and the orientation can be represented by Euler angles, quaternions, or rotation matrices, which reflects the external motion state of the robot interacting with the environment. This can be achieved by directly acquiring the real-time pose data of the end effector through the robot controller, or by tracking and measuring the end effector through an external vision system.

[0034] When identifying skill boundary points in the historical execution trajectory, these points refer to the moments or locations where one atomic skill action ends and another begins within a continuous historical execution trajectory. Identifying these boundary points helps decompose the continuous, complex robot behavior flow into discrete, meaningful atomic skill units, laying the foundation for subsequent segmentation and clustering. One approach is based on kinematic feature changes. Boundaries can be detected by analyzing significant changes in kinematic features such as joint velocity, acceleration, end effector velocity, and force / torque sensor data. For example, when a robot accelerates from a steady state or decelerates from a moving state to a stop, it usually indicates the start or end of a skill. Threshold detection, sliding window variance analysis, or machine learning-based anomaly detection algorithms can be used to identify these change points. Another approach is based on semantic information or expert annotation. During robot task execution, operator instructions or task phase information can be recorded synchronously. This information can be directly used as semantic annotations for skill boundaries. Alternatively, domain experts can manually mark the start and end points of skills by observing and analyzing historical trajectory data.

[0035] When segmenting the historical execution trajectory based on the skill boundary points to obtain multiple trajectory segments, segmentation refers to using the identified skill boundary points to precisely divide the original, continuous historical execution trajectory into a series of independent, complete sub-trajectories. Each sub-trajectory is a trajectory segment, representing a potential atomic skill action. Its function is to transform complex continuous behavioral flows into discrete units that can be independently analyzed and processed. One implementation method is direct slicing. Once the skill boundary points are identified, the original historical execution trajectory data can be sliced ​​directly at these time points, precisely dividing the data stream into multiple non-overlapping trajectory segments. Another implementation method is window-based extraction. Using each skill boundary point as the center, a certain time window is extended forward and backward to extract trajectory segments containing the boundary point and its preceding and following motion features. This method ensures that each segment contains complete skill transformation information.

[0036] When extracting kinematic and semantic features for each trajectory segment, kinematic features describe the physical properties of the robot's motion within the trajectory segment. Extracting these features quantifies the motion patterns of the trajectory segments for similarity comparison and clustering. One approach is to extract statistical features, such as the mean, variance, maximum, minimum, and peak values ​​of joint angles, joint velocities, joint accelerations, end effector positions, velocities, and accelerations within the trajectory segment. Another approach is to extract temporal features, such as dynamic time warping distance, Fourier transform coefficients, and wavelet transform coefficients, to characterize the overall shape and frequency characteristics of the trajectory. Semantic features describe the meaning or purpose of the action represented by the trajectory segment. Extracting these features helps to understand the function of the skill at a higher level, compensating for the shortcomings of pure kinematic features in distinguishing skills with similar functions but different motion details. One approach is to extract environmental interaction features; for example, in grasping actions, the type of object grasped, the grasping point position, and the grasping force can be extracted; in placement actions, the placement target area and placement posture can be extracted. This information can be obtained through robot sensors or task logs. The second approach involves extracting task context features, such as the higher-level task stage to which the trajectory segment belongs, the category of the operation object, and the purpose of the operation. These features can be extracted from the task description using natural language processing techniques or labeled using predefined tags.

[0037] When clustering the kinematic and semantic features to group similar trajectory segments into the same atomic skill category, clustering is an unsupervised learning method that aims to group objects in a dataset based on their similarity. Here, clustering's role is to group trajectory segments with similar kinematic and semantic features into one category, thereby forming categories representing specific atomic skills. An atomic skill category refers to a set of highly similar trajectory segments obtained after clustering. Each atomic skill category represents a basic action unit that the robot can independently execute, with a defined function and reusability. One implementation method is to use distance-based clustering algorithms, such as K-means, DBSCAN, or hierarchical clustering. In these algorithms, a suitable distance metric needs to be defined to measure the similarity between feature vectors of different trajectory segments. During clustering, kinematic and semantic features can be fused, and then the fused feature vectors are clustered. The second approach is to use deep learning-based clustering. This involves using deep learning models such as autoencoders or variational autoencoders to reduce the dimensionality of kinematic and semantic features and learn feature representations. Then, the learned low-dimensional embedding vectors are clustered. This method can automatically learn more discriminative feature representations, thereby improving the accuracy of clustering.

[0038] This application effectively solves the problem that inaccurate segmentation and clustering during the extraction of atomic skills from continuous historical execution trajectories can lead to atomic skills that do not conform to actual characteristics, affecting the construction of subsequent skill modules. Specifically, by explicitly acquiring historical execution trajectories containing joint angle sequences and end effector pose sequences, it ensures comprehensive coverage of the robot's motion state by the raw data, providing a high-quality foundation for subsequent analysis. Identifying skill boundary points allows continuous trajectories to be accurately decomposed into independent skill segments, avoiding segmentation errors caused by manual division or ambiguous definitions. Simultaneously extracting kinematic and semantic features for each trajectory segment enables a comprehensive representation of skill motion patterns and functional intentions, significantly improving the accuracy of distinguishing between different skill segments. Finally, clustering based on these multi-dimensional features aggregates truly similar trajectory segments into atomic skill categories that conform to the robot's actual execution capabilities, ensuring that the obtained atomic skill categories are accurate, independent, and reusable. This provides a solid and accurate foundation for subsequent parametric modeling of skill templates, encapsulation of atomic skill modules, and skill combination search during final task execution, greatly improving the efficiency and reliability of robot skill learning and reuse.

[0039] S2: Perform parametric modeling for each atomic skill category to obtain a skill template with a parameter interface.

[0040] Specifically, parametric modeling can extract the general motion patterns of a skill by averaging or interpolating all trajectory segments belonging to the same atomic skill category, and represent them as an adjustable template. The parameter interface can be simply defined as variable parameters such as keypoint coordinates, motion speed, or duration within this template. For example, for the atomic skill category of "movement," the starting point, intermediate point, and ending point can be extracted as parameters. When the robot needs to move from point A to point B, A and B are assigned to the starting and ending point parameters to generate the specific movement trajectory.

[0041] Please see Figure 4 , Figure 4 A specific implementation of step S2 is shown below: S21: Using dynamic motion primitives or probabilistic motion primitives, model all trajectory segments belonging to the same atomic skill category and fit them to generate the skill template. S22: Extract adjustable key parameters from the skill template and generate the parameter interface.

[0042] Dynamic Motion Primitives (DMPs) are a mathematical framework for representing and generating motion trajectories. They learn the shape of a demonstration trajectory and allow for generalization and adjustment with a small number of parameters. For example, a DMP typically consists of a gauge system and a nonlinear force term. The gauge system ensures the motion converges in time, while the nonlinear force term shapes the specific shape of the trajectory. The amplitude and target of the motion can be changed by adjusting the weights or target points of the force term. Furthermore, DMPs can approximate the nonlinear force term using linear combinations of Gaussian functions, making the learning and adjustment process more flexible and efficient. Probabilistic Motion Primitives (ProMPs), on the other hand, are a motion representation method based on probabilistic models. They treat the motion trajectory as a stochastic process and use probability distributions to describe the mean and covariance of the trajectory, thus representing the uncertainty and variability of the motion. For example, ProMPs typically use Gaussian processes or Gaussian mixture models to model the demonstration trajectory, learning the mean and covariance functions of the trajectory, which together define the probability distribution of the skill. In addition, ProMP can also represent the trajectory as a linear combination of basis functions through Bayesian linear regression and model the distribution of the basis function coefficients, thereby realizing a probabilistic representation of the motion trajectory.

[0043] By modeling all trajectory segments belonging to the same atomic skill category using dynamic motion primitives or probabilistic motion primitives and fitting them to generate skill templates, the aim is to extract the general motion patterns and core features of the atomic skill while retaining a certain degree of variability for subsequent parameter adjustments. For example, for DMP (Discretionary Motion Mastery), the demonstration trajectory data can be fitted into the DMP model using least squares or iterative optimization algorithms to learn the DMP parameters, thereby generating a skill template representing the atomic skill. For ProMP (Probability-Based Motion Mastery), the mean function and covariance function of ProMP can be learned from the demonstration trajectory data using maximum likelihood estimation or expectation-maximization algorithms, thus constructing a probabilistic skill template.

[0044] Adjustable key parameters are extracted from the generated skill template, and a parameter interface is generated. Adjustable key parameters are variables in the skill template that can be adjusted according to specific task requirements; they determine the specific details of skill execution. For example, for a "grab" skill, adjustable key parameters might include the three-dimensional spatial coordinates of the target object, the grabbing posture, the magnitude of the grabbing force, and the grabbing speed. For a "movement" skill, adjustable key parameters might include the starting position, target position, waypoint, movement speed, and obstacle avoidance distance. The parameter interface is a standardized mechanism exposed by the skill template to receive external parameter input. It allows the skill template to dynamically adjust its behavior according to different task contexts without modifying its internal logic. For example, the parameter interface can be a structure or class that defines the names, data types, and default values ​​of all adjustable key parameters, allowing external modules to assign values ​​when calling the skill. Furthermore, the parameter interface can also be provided in the form of an Application Programming Interface (API), allowing external systems to pass the required parameters to the skill template via function calls or message passing.

[0045] This application embodiment completes the parametric modeling of atomic skills by adapting the modeling method to the motion characteristics of robots. This not only ensures the accuracy of modeling the same type of atomic skills, but also realizes the parameterization and adjustment of skills, providing a basic support for the subsequent modular encapsulation and dynamic combination of atomic skills.

[0046] S3: Encode and encapsulate the skill template to generate atomic skill modules, and store the atomic skill modules in the skill library. Each atomic skill module records the preconditions and post-effects.

[0047] Specifically, encoding can be a simple hash of the skill template or assigning a unique identifier. Encapsulation involves packaging the skill template, parameter interface, and the skill's preconditions (e.g., "hand is free," "target object is in view") and post-effects (e.g., "hand holds object," "target object position changes") into a unified data structure. The skill library can be a simple list, array, or database for centralized management and storage of these atomic skill modules. For example, the "grab" skill module is encapsulated, and its preconditions could include "target object is reachable," "gripper is open," while its post-effects could include "target object is gripped," "gripper is closed."

[0048] Please see Figure 5 , Figure 5 A specific implementation of step S3 is shown below: S31: Train a neural network model using a contrastive learning approach to obtain a unified embedding encoder. S32: Map the skill templates to a vector space of a preset dimension using the unified embedding encoder, ensuring that semantically similar skill templates are close in distance within the vector space, thus obtaining an embedding vector corresponding to each skill template. S33: Encapsulate the skill template, the parameter interface, the embedding vector, and the preconditions and postconditions of the skill template to generate the atomic skill module, and store the atomic skill module in the skill library.

[0049] Specifically, a neural network model is trained using contrastive learning to obtain a unified embedding encoder. Contrastive learning is a self-supervised learning paradigm whose core idea is to learn effective feature representations by maximizing the similarity of similar sample pairs and minimizing the similarity of dissimilar sample pairs. Here, this approach is used to train a neural network model that can map different skill templates to a unified embedding space while maintaining semantic similarity within this space. For example, the SimCLR framework can be used to generate positive sample pairs by performing data augmentation operations such as synonym replacement and sentence restructuring on the descriptive text of the skill templates, and selecting other samples from the batch as negative samples, then training with the NT-Xent loss function. Alternatively, the InfoNCE loss function can be used, combined with pre-trained language models such as Transformer or BERT as the backbone network to encode the textual descriptions of the skill templates, and the model parameters can be optimized through a contrastive learning task to obtain an encoder capable of generating high-quality embedding vectors.

[0050] Subsequently, the skill templates are mapped to a vector space of a preset dimension using the unified embedding encoder described above. This ensures that semantically similar skill templates are close in distance within the vector space, thereby obtaining the embedding vector corresponding to each skill template. The unified embedding encoder is a trained neural network model whose function is to transform high-dimensional, complex skill template information (such as text descriptions, kinematic parameters, preconditions, etc.) into a fixed-length, low-dimensional numerical vector, i.e., the embedding vector. The preset-dimensional vector space means that the dimension of this embedding vector is predetermined, such as 128-dimensional, 256-dimensional, or 512-dimensional. This encoder can be a sequence encoder based on a recurrent neural network (RNN) or a long short-term memory network (LSTM) to process the serialized description of the skill templates and compress them into a fixed-dimensional context vector. Alternatively, the encoder can also be a Transformer-based model, such as BERT or RoBERTa. These models perform well in processing skill templates described in natural language, capturing complex semantic relationships and generating high-quality embedding vectors. Through comparative learning optimization, the encoder makes the Euclidean distance or cosine similarity between embedding vectors representing semantically similar skill templates smaller in the vector space, while the distance between embedding vectors representing semantically dissimilar skill templates is larger.

[0051] Finally, the skill template, parameter interface, embedding vector, and the preconditions and post-effects of the skill template are encapsulated to generate the atomic skill module, which is then stored in the skill library. Encapsulation integrates all relevant information of an atomic skill (including its core definition, adjustable parameters, semantic representation, and execution constraints) into a unified data structure or object, forming an independent, reusable software unit. For example, this information can be encapsulated into a JSON object or XML structure, containing a unique identifier for the skill template, the definition of the parameter interface (such as parameter name, type, default value, and value range), a numerical array of the embedding vector, and logical expressions for preconditions (such as robot state and environment state) and post-effects (such as changes in robot state and environment state). The skill library is a knowledge base that centrally stores and manages all atomic skill modules. It can be a relational database (such as MySQL or PostgreSQL) or a non-relational database (such as MongoDB or Redis) used to persistently store the encapsulated atomic skill modules so that they can be retrieved and used in subsequent task planning and execution processes.

[0052] This application embodiment trains a neural network model through contrastive learning to obtain a unified embedding encoder, which can learn the semantic differences between different skills, providing a foundation for the subsequent generation of a unified format skill representation, and avoiding the problem of being unable to compare similarity due to inconsistent coding standards for different skills.

[0053] S4: Receive the task description and parse the task description into a sequence of sub-targets.

[0054] Specifically, the task description can be text input by the user in natural language, such as "Please hand me the water glass." The parsing process can employ keyword matching, rule engines, or simple natural language processing techniques to decompose the high-level abstract task description into a series of specific, actionable sub-goals. For example, "hand me" can be identified as sub-goals such as "move to the user's vicinity" and "release the object." For instance, if the user inputs "Please put the book on the table onto the bookshelf," the system can parse it into sub-goals such as "grab the book," "move to the bookshelf," and "place the book."

[0055] S5: In the skill library, the atomic skill modules are combined and searched according to the sub-target sequence, the preconditions and the post-effects to generate a target skill chain.

[0056] Specifically, the search process can employ breadth-first search, depth-first search, or heuristic search algorithms. Starting from the initial state, based on the first sub-goal of the sub-goal sequence, the system searches the skill library for skill modules that satisfy the preconditions of that sub-goal. Once found, its post-goal effect is used as input for the next sub-goal, and the search continues until all sub-goals are satisfied, thus forming a complete skill execution sequence. For example, if the sub-goal sequence is "grab a book," "move to the bookshelf," and "place a book," the system will first search for the "grab" skill module, whose post-goal effect satisfies the precondition of "move to the bookshelf," then search for the "move" skill module, and so on.

[0057] Please see Figure 6 , Figure 6 A specific implementation of step S5 is shown below: S51: Using each atomic skill module in the skill library as a node, determine the directed edges between nodes based on the preconditions and post-effects to construct a directed skill graph. S52: Using the sub-target sequence as a search constraint, employ the A* algorithm to search the directed skill graph for a target skill chain with minimum cost from the initial state to the target state, wherein the cost is calculated based on at least one of execution time, energy consumption, or historical success rate.

[0058] Specifically, each atomic skill module in the skill library is treated as a node. Directed edges between nodes are determined based on the preconditions and post-effects to construct a directed skill graph. Each atomic skill module represents a basic operational unit that the robot can perform, such as "grasp," "move," or "place." When constructing the directed skill graph, these atomic skill modules are abstracted as discrete nodes in the graph. Preconditions refer to the state or environmental requirements that an atomic skill module must meet before execution; for example, the precondition for the "grasp" skill might be "the target object is within the graspable range." Post-effects refer to the state or environmental changes achieved after an atomic skill module completes execution; for example, the post-effect of the "grasp" skill might be "the target object has been grasped." If the post-effect of atomic skill module A satisfies the precondition of another atomic skill module B, a directed edge from A to B can be established between A and B, indicating that skill A can be followed by skill B after execution. In practice, the directed edges between nodes can be determined in various ways. For example, based on state matching, the system maintains a global state space, where the preconditions and post-effects of each atomic skill module are represented as state variables. A directed edge can be established when the set of post-effects of one skill has a non-empty intersection with or satisfies a relation with the set of preconditions of another skill. Alternatively, based on semantic matching, for more complex conditions and effects, natural language processing (NLP) techniques or ontology matching can be used to compare the semantics of preconditions and post-effects, determine their compatibility, and thus establish directed edges.

[0059] Based on this, using the sub-objective sequence as a search constraint, the A* algorithm is employed to search the directed skill graph for a minimum-cost target skill chain from the initial state to the target state. The sub-objective sequence is a series of ordered sub-objectives obtained after parsing the task description, such as "pick up the cup" -> "move to the table" -> "put down the cup." These sub-objectives define the intermediate states and the final goal of the task. During the search process, the A* algorithm prioritizes exploring paths that progressively satisfy the sub-objective sequence. For example, if the current sub-objective is "pick up the cup," the algorithm tends to select skill nodes whose post-effects include "the cup is picked up." The A* algorithm is a heuristic search algorithm that selects the next node to be expanded using the evaluation function "f(n) = g(n) + h(n)", where g(n) is the actual cost from the starting node to the current node n, and h(n) is the estimated cost from the current node n to the target node. In implementation, a priority queue can be used to store the nodes to be explored, and the node with the smallest f(n) value is taken from the queue for expansion each time. The initial state refers to the robot's current state before it performs the task, such as "the robot is idle, and the cup is on the table." The target state refers to the desired state after the task is completed, determined by the final sub-target of the sub-target sequence, such as "the cup is in the designated position." The A* algorithm starts from the skill node corresponding to the initial state and explores along the directed edges until it finds a skill sequence that can achieve the target state.

[0060] The cost is calculated based on at least one of execution time, energy consumption, or historical success rate. When calculating the cost of a skill chain, the average execution time of each atomic skill module can be pre-calibrated, and the execution times of all skills in the chain can be summed as part of the cost; alternatively, the execution time can be dynamically estimated based on skill parameters (e.g., movement distance, object size). Similarly, the average energy consumption of each atomic skill module can be pre-calibrated, and the energy consumption of all skills in the chain can be summed as part of the cost; alternatively, energy consumption can be dynamically estimated based on factors such as skill complexity and robot load. Furthermore, the system can record the success rate of each atomic skill module in historical executions. When calculating the cost of a skill chain, the reciprocal of the success rate (or some kind of penalty term based on the success rate) can be included as part of the cost; for example, the lower the success rate, the higher the cost. To more comprehensively evaluate the quality of the skill chain, a weighted sum can be used to combine multiple cost metrics.

[0061] This application embodiment structures the atomic skill modules in the skill base into a directed skill graph, clarifying the connection relationships between skills, thereby constructing a clear and efficient search space and effectively solving the problem of low search efficiency caused by the lack of a structured search framework.

[0062] S6: Perform timing coordination and parameter binding on the skill chain to generate an executable instruction sequence, and control the robot to execute the executable instruction sequence.

[0063] Specifically, timing coordination can arrange skills in a skill chain according to their logical order, ensuring that the execution order of skills conforms to task requirements. Parameter binding can populate the parameter interface of a skill template through user manual input, preset fixed values, or information extracted from the task context, thereby instantiating the abstract skill template into a specific execution action. For example, for the skills "grab a book" and "move to the bookshelf," timing coordination means performing the grab first, then the move. Parameter binding means assigning the specific spatial coordinates of the "book" to the parameters of the "grab" skill, and assigning the spatial coordinates of the "bookshelf" to the parameters of the "move" skill.

[0064] Please see Figure 7 , Figure 7 A specific implementation of step S6 is shown below: S61: Analyze the temporal dependencies between skill nodes in the target skill chain, determined by causal relationships and safety constraints. S62: Based on the temporal dependencies, insert transitional actions between sequentially executed skill nodes. S63: Perform pre-execution checks on parallel skill branches in the target skill chain. If a conflict is detected in the execution resources required by the parallel skill branch, convert the parallel execution to time-sharing sequential execution or insert a waiting operation. S64: After confirming no resource conflicts, obtain the spatial coordinates or attribute values ​​of specific objects in the task context corresponding to the task description using a visual sensor. S65: Assign the spatial coordinates or attribute values ​​to the parameter interface of the corresponding skill template in the target skill chain to generate a package-executable instruction sequence.

[0065] Specifically, this step involves analyzing the temporal dependencies between skill nodes in the target skill chain, determined by causal relationships and safety constraints. This step aims to clarify the execution order and interrelationships of each skill in the skill chain. Specifically, causal dependencies can be determined by analyzing the preconditions and postconditions of skills. For example, if the postcondition of skill A is a precondition of skill B, then skill A must be executed before skill B. Furthermore, safety constraints can be defined using pre-defined safety rules or domain knowledge. For example, it can stipulate that the gripper must be opened before a grasping action, or that the path must be unobstructed before moving to a specific area. Another approach is to use formal methods, such as Petri nets or state machines, to model the skill sequence, thereby automatically deriving the temporal dependencies between skills.

[0066] Based on this, this application inserts transitional actions between sequentially executed skill nodes according to the aforementioned temporal dependencies. This step aims to ensure smooth and safe transitions between different skills. For example, after a skill ends, the robot can perform a predefined "return to a safe pose" transitional action before starting the next skill. Alternatively, a trajectory interpolation algorithm can be used to generate a smooth transitional trajectory based on the end pose and velocity requirements of the preceding and following skills, avoiding abrupt changes in robot movements. Furthermore, typical transitional action patterns between different skill pairs can be extracted and generalized by learning from demonstration data and invoked during actual execution.

[0067] Simultaneously, this application performs pre-execution detection on parallel skill branches in the target skill chain. If a conflict is detected in the execution resources required by the parallel skill branches, the parallel execution is converted into time-sharing sequential execution or a waiting operation is inserted. This step aims to avoid execution failures or security issues caused by multiple skills simultaneously competing for limited resources. Specifically, a resource allocation table can be maintained to record the hardware resources (such as robotic arms, grippers, vision sensors, etc.) or software resources (such as specific algorithm modules) required by each skill. Before execution, the system checks whether parallel skill branches simultaneously request the same resource. If a conflict exists, one approach is to convert the conflicting parallel skills into time-sharing sequential execution, i.e., taking turns occupying resources at different time periods; another approach is to allow another skill to insert a waiting operation while one skill is executing, until the resource is released.

[0068] Furthermore, after confirming the absence of resource conflicts, this application acquires the spatial coordinates or attribute values ​​of specific objects within the task context corresponding to the task description using visual sensors. This step aims to obtain real-time, accurate information about key objects in the current task scenario to achieve dynamic adaptation of skill parameters. Specifically, the working environment can be perceived using visual sensors mounted on the robot (such as RGB cameras and depth cameras), and task-related target objects can be identified using image processing and computer vision algorithms (such as object detection, pose estimation, and semantic segmentation). The robot's three-dimensional spatial coordinates (such as position and pose) or attribute values ​​(such as color, size, and shape) in the robot's coordinate system can then be extracted. For example, in a grasping task, the visual sensor can identify the precise position and orientation of the object to be grasped.

[0069] Finally, this application assigns the spatial coordinates or attribute values ​​to the parameter interface of the corresponding skill template in the target skill chain, generating an executable instruction sequence. This step aims to instantiate the abstract skill template into specific instructions that the robot can directly execute. Specifically, each skill template has reserved adjustable parameter interfaces; for example, the grasping skill template may include parameters such as "target object position" and "grasping force." By filling these parameter interfaces with the actual spatial coordinates or attribute values ​​(such as the detected three-dimensional position of the cup) obtained by the vision sensor, the system can generate an executable instruction sequence containing specific values ​​for the current task scenario, such as "move to (x, y, z) and grasp with a force of F Newtons."

[0070] The embodiments of this application construct a complete process from timing analysis and conflict handling to parameter adaptation, ensuring that the combined skill chain can be executed stably and reliably by the robot, thereby improving the robot's adaptability and autonomy when facing complex and uncertain tasks.

[0071] Please see Figure 8 , Figure 8 A specific implementation method following step S6 is shown below in detail: S6A: Receive the execution status feedback from the sensors in real time, and compare the execution status with the expected status to obtain the execution result. S6B: When the execution result is detected as an execution failure, retry the current skill. If the number of retries for the same skill reaches a preset threshold and still fails, trigger replanning, generate an alternative skill chain based on the current status and the skill library that does not contain the current skill, and control the robot to execute the alternative skill chain.

[0072] Specifically, receiving real-time feedback from sensors aims to continuously acquire information about the actual situation during the robot's task execution. This execution status can include, but is not limited to, the robot's joint positions, speeds, torques, end effector poses, contact information with environmental objects, and scene images or depth information captured by visual sensors. This process can utilize various sensors integrated into the robot body, such as encoders, torque sensors, inertial measurement units (IMUs), and tactile sensors, to collect data in real time and transmit this data to the task execution monitoring module via the robot control system's data bus or communication interface. Alternatively, external sensors, such as visual sensors (e.g., RGB cameras, depth cameras, LiDAR) or auditory sensors installed in the working environment, can be used to monitor the robot and its manipulated objects in real time, transmitting the perceived data to the processing unit to parse the current execution status.

[0073] Subsequently, the executed state is compared with the expected state to obtain the execution result. This step is used to determine whether the robot's current skill execution meets expectations, i.e., whether it is successful. The expected state refers to the preset, ideal robot or environmental state before or during skill execution. The comparison process can be performed using preset thresholds or error ranges. For example, the real-time acquired end effector pose is compared with the expected target pose defined in the skill template using Euclidean distance. If the distance is less than a preset threshold, the expected state is considered to have been achieved; or the deviation between the joint angle sequence and the expected trajectory is compared. If the deviation is within the allowable range, the execution is considered successful. Another approach is to use a model-based prediction method to construct a predictive model for skill execution, predict the subsequent state based on the current execution state, and compare it with the actual received execution state. If there is a significant deviation between the prediction and the actual state, it may indicate execution failure.

[0074] When an execution failure is detected, the system will first retry the current skill. This mechanism provides preliminary fault tolerance, designed to handle occasional, non-systematic execution failures. Upon receiving a failure signal, the robot control system can simply resend the executable instruction sequence corresponding to the current skill and restart the skill execution process. Before retrying, a brief pause can be made to ensure the robot returns to a relatively safe or initial state. Alternatively, minor adjustments can be made to the robot or environment before retrying; for example, if grasping fails, the grasping pose or force can be fine-tuned before re-executing the grasping skill to increase the probability of a successful retry.

[0075] If the number of retries for the same skill reaches a preset threshold without success, a re-planning is triggered. The system maintains a counter, which increments each time the current skill is retried. When the counter reaches a preset integer threshold (e.g., 2 or 3 times), the system issues a re-planning command. This threshold can also be based on time or resource consumption; for example, if multiple retries within a certain time period fail, or if the energy or time consumed by the retries exceeds a certain limit, a re-planning is triggered.

[0076] After triggering a replanning phase, the system generates an alternative skill chain that does not contain the current skill, based on the current state and the skill library. The replanning module can utilize the same search algorithm as the initial planning phase, such as the A* algorithm, but updates the starting state of the search to the robot's current actual state. Skills that lead to failure are temporarily removed from the available skill set or given extremely high costs. Then, the system searches the directed skill graph again for a path from the current state to the target state, generating a new skill chain. Alternatively, methods based on reinforcement learning or heuristic search can be used to intelligently select alternative skills by learning from historical failures and successful replanning experiences. For example, if "grabbing object A" fails, the system might try "pushing object A" or "grabbing object A using a suction cup" (if these alternative skills exist in the skill library), or search for a skill chain that bypasses object A to directly complete subsequent tasks.

[0077] Finally, the robot is controlled to execute the alternative skill chain. After the reprogramming module generates the alternative skill chain, it passes it to the instruction sequence execution module. This module performs timing coordination and parameter binding on the alternative skill chain, just as it did with the initial target skill chain, generating a new executable instruction sequence and driving the robot to execute it. Before executing the alternative skill chain, the system can also perform a pre-execution simulation or safety check to ensure that the new skill chain is safe and feasible in the current environment.

[0078] The embodiments of this application solve the problem that the task terminates when the original solution encounters a single-step skill failure, which greatly improves the success rate and robustness of the robot in performing tasks in complex and uncertain environments, and ensures that the robot can efficiently and reliably complete complex combined tasks even when facing unknown environments and uncertainties.

[0079] This application provides an embodiment of a nursing home companion robot. The scenario involves a nursing home deploying a dual-arm wheeled companion robot to provide personalized services to different elderly residents. The robot has accumulated 10 atomic skill modules through prior learning and stored them in a skill library.

[0080] First, the skill library is constructed (initially offline phase): The skill abstraction module extracts the "grab" skill template from 50 demonstration trajectories of "grabbing different objects". Parameter interfaces include: target object 3D position, grab height (default 0.05m), and gripping force (default 2N). Motion primitive parameters have been fitted. Similarly, 10 atomic skills were extracted: "move", "place", "clean the desktop", "deliver", "press (for pressing elevator buttons)", "rotate (for unscrewing bottle caps)", "classify (place different medicines into corresponding compartments)", "lift", and "put down". The skill encoding module encodes these 10 skills into 10 256-dimensional embedding vectors and stores them in the skill library. The cosine similarity between the embedding vectors of "lift" and "place" is 0.32 (relatively far), while that between "grab" and "lift" is 0.78 (relatively close), consistent with semantic intuition.

[0081] Then, a new task combination will be performed: medication retrieval and delivery. Task description: "Please retrieve the blood pressure medication from the medicine cabinet in room A and deliver it to Grandma Zhang in room 302." The task parser outputs the following sub-target sequence: [Move to the medicine cabinet, identify the antihypertensive medication, grab the medicine bottle, move to room 302, and deliver it to Grandma Zhang].

[0082] Skill combination engine performs graph search: 1. Node matching: Matches "Move" skill (with target room parameters), "Grab" skill, and "Deliver" skill from the skill library.

[0083] 2. Inspection during the process: The transition from "grabbing" to "moving" must meet the condition that "the robotic arm is in a safe position after the gripping is completed." The system automatically inserts the "lifting" skill (lifting the medicine bottle to a height of 0.2m) between the two.

[0084] 3. The final skill chain is: Move (medicine cabinet) → Grab (blood pressure medicine) → Lift (height=0.2m) → Move (room 302) → Deliver (Grandma Zhang).

[0085] 4. Planner calculates total cost: estimated execution time 35 seconds, overall score 0.92 (out of 1.0).

[0086] Next, execution scheduling is completed. The timing coordinator resolves dependencies: all skills are executed sequentially, with transition actions automatically generated. For example, "lifting" is automatically triggered after "grabbing," without explicit programming. The parameter binder: visually identifies the location coordinates of the medicine cabinet (x=2.3m, y=1.8m) and binds them to the target parameters of "moving"; it also identifies the 3D coordinates of the blood pressure medication bottle (x=2.35m, y=1.82m, z=0.75m) and binds them to the position parameters of "grabbing."

[0087] Execution begins: 1. Movement Skill: The robot navigates to the medicine cabinet in 12 seconds.

[0088] 2. Grasping skill: The robotic arm extends and grips the medicine bottle. The gripping force sensor feedback reaches 1.9N (target 2N±0.3N), and the judgment is successful. The time taken is 3 seconds.

[0089] 3. Lifting Skill: Raises the medicine bottle to a safe height of 0.2m in 1 second.

[0090] 4. Movement skill: Navigate to room 302, taking 15 seconds.

[0091] 5. Delivery Skill: The robotic arm extends the medicine bottle 0.3m in front of Grandma Zhang, with a voice prompt, "Grandma Zhang, your blood pressure medication," taking 4 seconds. Complete.

[0092] Alternatively, parallel combinations can be performed: simultaneously cleaning and fetching medicine. Task description: "Clean up clutter on the table and fetch a glass of water from the kitchen." The task parser identifies two parallel sub-objectives: [cleaning the table] and [fetching water]. The combination engine checks resources: cleaning the table uses the right hand (holding a rag), fetching water uses the left hand (holding a glass), and moving the chassis is a shared resource, so they cannot truly move simultaneously. Engine output solution: Adopting a "time-sharing parallel" strategy, i.e., the chassis moves to the kitchen first (right hand idle, left hand fetches water), and after fetching water, the chassis moves to the table (left hand holds the glass, right hand performs cleaning). The actual generated combination sequence is: [move (kitchen), grab (glass, left hand), move (tableside), clean the table (right hand, rag)], with the two main actions overlapping in time (both hands can be pre-positioned during movement). The execution scheduling module coordinates the movement planning of the left and right arms in parallel to avoid collisions. The final total time is 18 seconds, while sequential execution takes 28 seconds, resulting in a 36% efficiency improvement.

[0093] Finally, in the abnormal recovery scenario: During the "grabbing" step of the "medicine delivery" task, a visual positioning error caused the gripper to shift, and the gripping force sensor returned 0N (the medicine bottle was not grasped). The abnormal recovery processor detected the failure and triggered a retry: visual positioning was re-performed (the position of the medicine bottle was re-identified), the grasping posture was adjusted, and the attempt was made again. The second grasp was successful, but the time was increased by an additional 2 seconds. If three consecutive retries fail, the processor will call the combinatorial engine to replan: generate an alternative skill chain [move to the medicine cabinet, request manual assistance (voice call to a caregiver)], and the caregiver will manually retrieve the medicine and hand it over to the robot to complete the subsequent delivery.

[0094] The nursing home companion robot described in this application can combine existing skills like "building blocks" to quickly adapt to the diverse service needs of the elderly without the need for engineers to reprogram it. Compared to traditional methods (where each new task requires several days to weeks of data collection and model training), this application reduces the deployment time of new tasks to minutes, significantly improving the practical value of the robot in dynamic elderly care environments.

[0095] Please refer to Figure 9 As a response to the above Figure 2 The implementation of the method shown in this application provides an embodiment of a task execution device based on skill modularization, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various robots.

[0096] like Figure 9 As shown, the task execution device based on skill modularization in this embodiment includes: a historical execution trajectory acquisition module 71, a parametric modeling module 72, a skill template encapsulation module 73, a task description parsing module 74, a target skill chain generation module 75, and an instruction sequence execution module 76, wherein: The historical execution trajectory acquisition module 71 is used to acquire the robot's historical execution trajectory and to segment and cluster the historical execution trajectory to obtain multiple atomic skill categories; Parametric modeling module 72 is used to perform parametric modeling on each atomic skill category to obtain a skill template with a parameter interface; The skill template encapsulation module 73 is used to encode and encapsulate the skill template, generate atomic skill modules, and store the atomic skill modules in the skill library. Each atomic skill module records the preconditions and post-effects. Task description parsing module 74 is used to receive a task description and parse the task description into a sub-target sequence; The target skill chain generation module 75 is used to perform a combined search on the atomic skill modules in the skill library according to the sub-target sequence, the preconditions and the post-effects to generate a target skill chain; The instruction sequence execution module 76 is used to perform timing coordination and parameter binding on the skill chain, generate an executable instruction sequence, and control the robot to execute the executable instruction sequence.

[0097] Furthermore, the historical execution trajectory acquisition module 71 includes: An execution trajectory acquisition unit is used to acquire the robot's historical execution trajectory, wherein the historical execution trajectory includes a joint angle sequence and an end effector pose sequence; The skill boundary point identification unit is used to identify skill boundary points in the historical execution trajectory; The trajectory segmentation unit is used to segment the historical execution trajectory based on the skill boundary points to obtain multiple trajectory segments; A feature extraction unit is used to extract kinematic and semantic features from each trajectory segment; The feature clustering unit is used to cluster the kinematic features and the semantic features to aggregate similar trajectory segments into the same atomic skill category.

[0098] Furthermore, the parametric modeling module 72 includes: The modeling unit is used to model all trajectory segments belonging to the same atomic skill category using dynamic motion primitives or probabilistic motion primitives, and fit and generate the skill template. The parameter interface generation unit is used to extract adjustable key parameters from the skill template and generate the parameter interface.

[0099] Furthermore, the skill template encapsulation module 73 includes: The encoder generation unit is used to train the neural network model through contrastive learning to obtain a unified embedding encoder. An embedding vector generation unit is used to map the skill template to a vector space of a preset dimension through the unified embedding encoder, so that semantically similar skill templates are close in distance in the vector space, and to obtain an embedding vector corresponding to each skill template. The encapsulation unit is used to encapsulate the skill template, the parameter interface, the embedding vector, and the preconditions and postconditions of the skill template to generate the atomic skill module, and store the atomic skill module in the skill library.

[0100] Furthermore, the target skill chain generation module 75 includes: A directed skill graph construction unit is used to construct a directed skill graph by taking each atomic skill module in the skill library as a node, determining the directed edges between nodes according to the preconditions and the postconditions; The skill chain search unit is used to search for a target skill chain with the minimum cost from the initial state to the target state in the directed skill graph using the sub-target sequence as a search constraint and the A* algorithm. The cost is calculated based on at least one of execution time, energy consumption, or historical success rate.

[0101] Furthermore, the instruction sequence execution module 76 includes: The parsing unit is used to parse the temporal dependencies between skill nodes in the target skill chain, which are determined by causal relationships and security constraints. A transition action insertion unit is used to insert transition actions between sequentially executed skill nodes according to the temporal dependency relationship; The conflict detection unit is willing to perform pre-execution detection on the parallel skill branches in the target skill chain. If a conflict is detected in the execution resources required by the parallel skill branches, the parallel execution will be converted into time-sharing sequential execution or a waiting operation will be inserted. The object information acquisition unit is used to acquire the spatial coordinates or attribute values ​​of specific objects in the task context corresponding to the task description through a visual sensor after confirming that there are no resource conflicts. The assignment unit is used to assign the spatial coordinates or attribute values ​​to the parameter interface of the corresponding skill template in the target skill chain, thereby generating a package executable instruction sequence.

[0102] Furthermore, the instruction sequence execution module 76 also includes: The execution result acquisition module is used to receive the execution status feedback from the sensor in real time, and compare the execution status with the expected status to obtain the execution result; The alternative skill chain generation module is used to retry the current skill when the execution result is detected as failure. If the number of retries for the same skill reaches a preset threshold and still fails, a replanning is triggered to generate an alternative skill chain that does not contain the current skill based on the current state and the skill library, and the robot is controlled to execute the alternative skill chain.

[0103] It should be noted that any AI models, software tools, or components not belonging to this company appearing in the embodiments of this application are merely illustrative examples and do not represent actual use. All user personal information involved in the embodiments of this application has been obtained by authorized entities (who have known and consented) or fully authorized by all parties through various legal and compliant means. The collection, storage, use, processing, transmission, provision, and disclosure of the information, data, and signals involved all comply with relevant laws and regulations and do not violate public order and good morals.

[0104] To address the aforementioned technical problems, embodiments of this application also provide a robot. Please refer to [link / reference needed] for details. Figure 10 , Figure 10 This is a basic structural block diagram of the robot in this embodiment.

[0105] Robot 8 includes a memory 81, a processor 82, and a network interface 83 that are interconnected via a system bus. It should be noted that... Figure 10Only a robot 8 with three components—memory 81, processor 82, and network interface 83—is shown. It should be understood that implementing all shown components is not required; more or fewer components can be implemented alternatively. Those skilled in the art will understand that the robot described herein is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), and embedded devices.

[0106] Robots can be computing devices such as desktop computers, laptops, PDAs, and cloud servers. Robots can interact with users through keyboards, mice, remote controls, touchpads, or voice-controlled devices.

[0107] The memory 81 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 81 may be an internal storage unit of the robot 8, such as the hard disk or memory of the robot 8. In other embodiments, the memory 81 may also be an external storage device of the robot 8, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the robot 8. Of course, the memory 81 may also include both internal storage units and external storage devices of the robot 8. In this embodiment, the memory 81 is typically used to store the operating system and various application software installed on the robot 8, such as program code for task execution methods based on skill modularization. In addition, the memory 81 may also be used to temporarily store various types of data that have been output or will be output.

[0108] In some embodiments, processor 82 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. This processor 82 is typically used to control the overall operation of robot 8. In this embodiment, processor 82 is used to run program code stored in memory 81 or process data, for example, to run the program code of the skill-modular task execution method described above, to implement various embodiments of the skill-modular task execution method.

[0109] The network interface 83 may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the robot 8 and other electronic devices.

[0110] This application also provides another embodiment, namely, providing a computer-readable storage medium storing a computer program that can be executed by at least one processor to cause the at least one processor to perform the steps of the skill-modular task execution method described above.

[0111] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods of the various embodiments of this application.

[0112] Obviously, the embodiments described above are merely some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the scope of this application. This application can be implemented in many different forms; rather, these embodiments are provided to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of protection of this application.

Claims

1. A task execution method based on skill modularization, characterized in that, include: The robot's historical execution trajectory is obtained, and the historical execution trajectory is segmented and clustered to obtain multiple atomic skill categories; Parametric modeling is performed on each atomic skill category to obtain a skill template with a parameter interface; The skill template is encoded and encapsulated to generate atomic skill modules, and the atomic skill modules are stored in the skill library. Each atomic skill module records the preconditions and post-effects. Receive a task description and parse the task description into a sequence of sub-targets; In the skill library, the atomic skill modules are combined and searched according to the sub-target sequence, the preconditions, and the post-effects to generate a target skill chain; The skill chain is time-coordinated and parameter-bound to generate an executable instruction sequence, and the robot is controlled to execute the executable instruction sequence.

2. The task execution method based on skill modularization as described in claim 1, characterized in that, The process involves acquiring the robot's historical execution trajectory, segmenting and clustering the historical execution trajectory to obtain multiple atomic skill categories, including: The robot's historical execution trajectory is obtained, wherein the historical execution trajectory includes a joint angle sequence and an end effector pose sequence; Identify the skill boundary points in the historical execution trajectory; The historical execution trajectory is segmented based on the skill boundary points to obtain multiple trajectory segments; Extract the kinematic and semantic features from each trajectory segment; Clustering is performed on the kinematic features and the semantic features to aggregate similar trajectory fragments into the same atomic skill category.

3. The task execution method based on skill modularization as described in claim 1, characterized in that, The parametric modeling of each atomic skill category yields a skill template with a parameter interface, including: Using dynamic motion primitives or probabilistic motion primitives, model all trajectory segments belonging to the same atomic skill category, and fit them to generate the skill template; Adjustable key parameters are extracted from the skill template to generate the parameter interface.

4. The task execution method based on skill modularization as described in claim 1, characterized in that, The process of encoding and encapsulating the skill template to generate atomic skill modules, and storing the atomic skill modules in the skill library, includes: A unified embedding encoder is obtained by training a neural network model using a contrastive learning approach. The unified embedding encoder maps the skill templates to a vector space of a preset dimension, so that semantically similar skill templates are close in distance in the vector space, thus obtaining the embedding vector corresponding to each skill template. The skill template, the parameter interface, the embedding vector, and the preconditions and postconditions of the skill template are encapsulated to generate the atomic skill module, and the atomic skill module is stored in the skill library.

5. The task execution method based on skill modularization as described in claim 1, characterized in that, The step of combining and searching the atomic skill modules in the skill library based on the sub-target sequence, the preconditions, and the post-effects to generate a target skill chain includes: Each atomic skill module in the skill library is used as a node, and directed edges between nodes are determined according to the preconditions and post-effects to construct a directed skill graph. Using the sub-target sequence as a search constraint, the A* algorithm is used to search for a target skill chain from the initial state to the target state in the directed skill graph with the minimum cost, wherein the cost is calculated based on at least one of execution time, energy consumption, or historical success rate.

6. The task execution method based on skill modularization as described in claim 1, characterized in that, The step of performing timing coordination and parameter binding on the skill chain to generate an executable instruction sequence, and controlling the robot to execute the executable instruction sequence, includes: Analyze the temporal dependencies between skill nodes in the target skill chain, determined by causal relationships and security constraints; Based on the temporal dependencies, transition actions are inserted between sequentially executed skill nodes; Before execution, a pre-execution detection is performed on the parallel skill branches in the target skill chain. If a conflict is detected in the execution resources required by the parallel skill branches, the parallel execution is converted into time-sharing sequential execution or a waiting operation is inserted. After confirming that there are no resource conflicts, the spatial coordinates or attribute values ​​of the specific objects in the task context corresponding to the task description are obtained through a visual sensor. The spatial coordinates or attribute values ​​are assigned to the parameter interface of the corresponding skill template in the target skill chain to generate a package executable instruction sequence.

7. The task execution method based on skill modularization as described in any one of claims 1 to 6, characterized in that, After performing timing coordination and parameter binding on the skill chain to generate an executable instruction sequence, and controlling the robot to execute the executable instruction sequence, the method further includes: The system receives the execution status feedback from the sensors in real time and compares the execution status with the expected status to obtain the execution result. When the execution result is detected as failure, the current skill is retried. If the number of retries for the same skill reaches a preset threshold and still fails, a replanning is triggered. Based on the current state and the skill library, an alternative skill chain that does not contain the current skill is generated, and the robot is controlled to execute the alternative skill chain.

8. A task execution device based on skill modularization, characterized in that, include: The historical execution trajectory acquisition module is used to acquire the robot's historical execution trajectory, and to segment and cluster the historical execution trajectory to obtain multiple atomic skill categories; The parametric modeling module is used to perform parametric modeling on each atomic skill category to obtain skill templates with parameter interfaces; A skill template encapsulation module is used to encode and encapsulate the skill template to generate atomic skill modules and store the atomic skill modules in the skill library. Each atomic skill module records the preconditions and post-effects. The task description parsing module is used to receive a task description and parse the task description into a sub-target sequence. The target skill chain generation module is used to perform a combined search on the atomic skill modules in the skill library based on the sub-target sequence, the preconditions, and the post-effects to generate a target skill chain; The instruction sequence execution module is used to perform timing coordination and parameter binding on the skill chain, generate an executable instruction sequence, and control the robot to execute the executable instruction sequence.

9. A robot comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the task execution method based on skill modularization as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the task execution method based on skill modularization as described in any one of claims 1 to 7.

Images