A method and device for generating embodied intelligent robot motions based on the WASM large model

By proposing an embodied intelligent robot motion generation method based on the WASM large model, and using a conditional diffusion model to fit the robot task execution trajectory, the problem of robot task storage and reuse in the existing technology is solved, and efficient, low-computing-power and continuous trajectory generation is achieved.

CN122914182BActive Publication Date: 2026-11-17SHANGHAI SAGE INTELLIGENT TECH CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202611417441.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2026-09-14
Publication Date
2026-11-17
Estimated Expiration
2046-09-14

AI Technical Summary

Technical Problem

Existing technologies struggle to effectively store and reuse robot task execution experience, especially in new task scenarios where it is difficult to efficiently match and generate continuous, highly cohesive trajectories. Furthermore, they suffer from problems such as skill combination explosion and high computational consumption.

Method used

We adopt an embodied intelligent robot motion generation method based on the WASM large model. By encapsulating the complete task execution process into a line library, we use a conditional diffusion model to fit the lines and generate motion trajectories that fit the current task, thus avoiding the traditional atomic skill library solution and massive skill retrieval.

Benefits of technology

It achieves efficient storage and reuse of experience, maintains the temporal integrity and physical continuity of action sequences, reduces inference computing power consumption, and realizes the representation and trajectory generation of new tasks with a small number of initial operations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122914182B_ABST
    Figure CN122914182B_ABST
Patent Text Reader

Abstract

This invention provides a method and device for generating embodied intelligent robot movements based on a WASM large-scale model, relating to the field of embodied intelligent robot control technology. The method includes: encapsulating each complete task execution process of the robot as lines, which are then stored in a database after representative screening; generating anchor point lines through 1-3 initial operations for new tasks, retrieving and matching multiple similar lines as prior condition inputs, and fitting them to a WASM large-scale model with a conditional diffusion model architecture; and generating the movement trajectory through inverse denoising. This invention uses complete trajectories as the basic storage unit, ensuring a constant and controllable database size, avoiding skill combination explosion, maintaining the integrity of the temporal sequence and physical continuity of movements, and achieving constant-level retrieval overhead, thus realizing intelligent generation of robot movements with low computational power, few samples, and self-evolutionary capabilities.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of embodied intelligent robot control technology, specifically to an intelligent generation method for embodied robot motion based on matching and fitting a WASM large model with a task execution trajectory line library, and more particularly to an embodied intelligent robot motion generation method and device based on a WASM large model. Background Technology

[0002] The acquisition and reuse of existing robot operation skills mainly rely on methods such as teaching programming, atomic skill library combination and invocation, world model prediction, and diffusion model trajectory generation. Teaching programming is inefficient and difficult to transfer across tasks; atomic skill library schemes (such as the invention patent with publication number WO2023273178A1) deconstruct actions into the smallest skill units and then combine them, which faces a serious combinatorial explosion problem as the number of skills increases, and breaking down complete operations into independent skill units destroys the temporal coupling and physical continuity between actions; world model methods (such as AdaWorld and Dreamer) have environmental prediction capabilities, but lack explicit storage and efficient reuse mechanisms for historical complete task experience; diffusion model trajectory generation methods (such as DRIFT and CTD) can generate trajectories that meet constraints, but it is difficult to directly use the prior information of multiple historical trajectories to accelerate the generation process, and each new task still requires complete iterative reasoning.

[0003] In summary, existing technologies have not yet solved the following core problems: 1) How to store the robot's complete task execution experience in a continuous and highly cohesive form; 2) How to achieve efficient matching with the experience base using a small number of real operation samples in new task scenarios; 3) How to perform intelligent fitting and generalization at the trajectory level based on multiple matched experiences, avoiding the computational power consumption caused by massive fine-grained skill retrieval; 4) How to maintain the temporal integrity and physical continuity of the action sequence in the above process. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a method and device for generating embodied intelligent robot motions based on the WASM large model.

[0005] According to the present invention, a method and device for generating embodied intelligent robot motion based on the WASM large model are provided, the solution of which is as follows: Firstly, a method for generating embodied intelligent robot actions based on the WASM large model is provided, the method comprising: Acquire multi-source data of the entire process of the robot's historical task execution, encapsulate the continuous key execution action data chain corresponding to each complete task execution process into candidate lines, perform representative screening on the candidate lines, and store the screened representative lines into a line library, with the complete task execution trajectory as the basic storage unit for the lines. When a new task is received, the entire process data is collected through at least one initial operation and encapsulated into anchor line. The attribute feature vector of the anchor line is then extracted as a query vector. The query vector is used to search the line library and match several similar lines. The matched similar lines are used as prior conditions and input into the WASM large model with a conditional diffusion model architecture for line fitting. The motion trajectory adapted to the current task is generated through the reverse denoising process. The motion trajectory is converted into a sequence of control instructions that the robot can execute and then executed.

[0006] Preferably, the multi-source data includes: kinematic trajectory data, force data, visual state data, task parameter data, and temporal structure data; The kinematic trajectory data includes: a complete time series of joint angles, angular velocities, and angular accelerations; The force data includes: continuous curves showing the magnitude and direction of the contact force / torque at each joint and the contact force at the end effector; The visual state data includes: the temporal sequence of the operation scene image and the pose change trajectory of the operation object; The task parameter data includes: task type identifier, geometric and physical properties of the operation object, and environmental constraint parameters; The time-series structure data includes: the total duration of the task and the time boundaries of each operation stage; When encapsulating lines, a multimodal fusion method is used to identify the boundaries of key stages in the task execution process based on force abrupt changes and visual state changes, and to extract key parameterized features of each stage; The key parameterized features include displacement, rotation angle, velocity curve shape and force control curve shape. The original time series data is lossily compressed to retain key feature points in extreme points, inflection points and abrupt change points, and the lines are encapsulated into a standardized data structure. The data structure includes fields such as task type label, operation object attribute vector, scene attribute vector, key stage sequence, compressed trajectory data, execution result label, and statistical weight.

[0007] Preferably, the representative screening includes: calculating the comprehensive feature similarity between the candidate lines and representative lines already stored in the line library under the same task type. Sim(t candidate , t stored ) = cos(f candidate , f stored ) Where, τ candidate Indicates candidate lines, τ stored f represents the representative lines that have been stored. candidate f represents the comprehensive attribute feature vector of the candidate lines. stored This represents the comprehensive attribute feature vector of the stored representative lines. Sim represents the comprehensive feature similarity; cos() represents the cosine similarity, which measures the directional similarity between two feature vectors in the feature space. If Sim > θ high If a candidate line is determined to be a redundant expression based on existing experience, it is rejected from being added to the database, and the statistical weight w←w+1 of the representative line is updated. If θ low <Sim≤θ high If a candidate line is determined to contain new environmental features or operational fine-tuning, it is stored as a new representative line in the line library. If Sim≤θ low This is determined to be a new task type and is directly added to the database as the first representative line of the new category; among them, θ high For high similarity threshold, θ low For low similarity threshold, θ high The value range of θ is 0.92 to 0.98. low The value ranges from 0.65 to 0.80, where w represents the number of successful executions of the category represented by the representative line.

[0008] Preferably, when the number of representative lines for a certain task type in the line library reaches a preset upper limit N. max At this time, the line merging mechanism is triggered: Perform K-means clustering on all representative lines under this task type, with N clusters. max / 2, after clustering, multiple lines within each category are merged into a central line. The trajectory data of the central line is the weighted average of the trajectories within the category, the attribute feature vector is the center within the category, the statistical weights are accumulated, and the redundant individuals that are merged are deleted. Where, N max N is the preset upper limit for the number of representative lines under a single task type. max The value range is 5 to 15; When data is added to the database, a multi-dimensional index is constructed simultaneously. The multi-dimensional index includes a classification index based on task type labels and a vector index based on attribute feature vectors. After the task is completed, the execution trajectory is used as a representative candidate line for the selection process, thereby enabling dynamic updating and streamlined maintenance of the line library.

[0009] Preferably, the at least one initial operation is 1 to 3 initial operations performed through any one of the following methods: teaching operation, programming execution, video imitation learning, or remote operation.

[0010] Preferably, the retrieval includes: a coarse screening based on task type labels for classification index filtering, retrieving only subsets of lines with the same or similar task types; and a vector similarity retrieval based on attribute feature vectors on the coarse screening results. Sim(f anchor , f i ) = f anchor ·f i / (‖f anchor ‖·‖f i ‖) Then, the candidate lines are comprehensively scored from multiple dimensions: Score = α·Sim attr + b·Sim scene + c·Sim structure + d·Quality The K historical lines with the highest overall scores are selected as the matching results, where K ≥ 2; f anchor The attribute feature vector representing the anchor point line. f i Let represent the attribute feature vector of the i-th candidate line, where i represents the index of the candidate line, and ||·|| represents the magnitude of the vector. Sim attr Indicates attribute similarity. Sim scene Indicates scene matching degree. Sim structure Indicates the degree of matching of the stage structure. Quality The historical execution quality score is represented by α, β, γ, and δ, which are the score weight coefficients, and α+β+γ+δ=1. K represents the number of similar lines matched.

[0011] The line fitting is achieved through the trajectory encoder, conditional encoder, conditional diffusion model, and trajectory decoder of the WASM large model; The trajectory encoder, based on the Transformer architecture, encodes variable-length line temporal trajectory data into a fixed-dimensional latent state vector: z i = Enc traj (ti ),z i ∈ R d Where, τ i Let represent the trajectory sequence of the i-th input line. Enc traj Represents the trajectory encoding function. z i Let represent the hidden state vector of the trajectory of the i-th input line. R d Let represent a d-dimensional real vector space, where d represents the dimension of the latent space; The conditional encoder includes a priori distribution coding module, an anchor point constraint coding module, and a fusion output module; The prior distribution coding module extracts statistical priors from the K matching lines: c prior = Aggregate(z 1 , z 2 , …, z K ) The aggregation method employs mean pooling or weighted fusion based on cross-attention, with the weights of the weighted fusion and the overall score of each matched line being calculated. i Proportional; The anchor point constraint encoding module encodes the key stage boundaries and key parameters in the anchor point lines into constraint condition vectors. c constraint The constraint vector includes the starting pose, ending pose, necessary path points, force control curve envelope, and velocity limit; the fusion output module fuses the prior, constraint, and anchor point attributes into a comprehensive condition vector. c = Fuse(c prior , c constraint , f anchor ) in, c prior Represents the statistical prior vector. Aggregate Represents aggregate functions, z1, z2, ..., z K Score represents the hidden state vector of the trajectory of the first to the Kth matching lines. i This represents the overall score for the i-th matching line. c constraint Represents the constraint vector. f anchor The attribute feature vector representing the anchor point line. cThis represents the combined condition vector obtained by fusion, and Fuse represents the fusion function.

[0012] Preferably, the conditional diffusion model adopts a denoised diffusion probability model architecture; Gaussian noise is gradually added to the target trajectory τ0 during the forward diffusion process:

[0013] The reverse denoising process, guided by the comprehensive condition vector c, gradually denoises from pure noise to... :

[0014] The denoising network employs a Diffusion Transformer (DiT) architecture, taking the trajectory sequence tokens and the comprehensive conditional vector as input to predict the added noise. Furthermore, the denoising process utilizes a classifier-independent guidance strategy.

[0015] The single-step update formula is:

[0016] in, Indicates the target trajectory. and Representing the t-th step and the t-th step respectively The noisy trajectory in step 1, where t represents the diffusion time step. Indicates by Forward noise addition The conditional probability distribution, where N represents a Gaussian distribution. Here, I represents the preset noise scheduling coefficients. Indicated by For conditions by Denoising The conditional probability distribution, and Let represent the mean and variance of the predictions made by the denoising network, respectively, and θ represent the parameters of the denoising network. This represents the noise predicted by the denoising network. Indicates an empty condition. For guiding strength coefficient, >1, This represents the noise prediction value after correction by classifier-independent guidance. =1 , t =α1α2…α t That is, the first t steps α s The product of these products, where s is the diffusion time step number. Let be the posterior variance, and z be random noise following a standard Gaussian distribution N(0, I); the diffusion step number T ranges from 50 to 200. The value range is 2 to 8.

[0017] Preferably, M candidate trajectories are sampled from the conditional distribution, where M ≥ 3, and the optimal trajectory is selected based on the following objective function: t = argmin τ [l 1 ·L smooth (t) + λ 2 ·L constraint (t) + λ 3 ·L energy (τ)], τ ∈ {τ(1), τ(2), …, τ(M)} Where, τ This indicates the selected optimal trajectory. argmin τ This represents the trajectory selection operation that minimizes the objective function from the set of candidate trajectories. τ(1), τ(2), …, τ(M) Let λ1, λ2, and λ3 represent the candidate trajectories from the 1st to the Mth, where M represents the number of sampled candidate trajectories, and λ1, λ2, and λ3 are the weight coefficients of each cost. L smooth Cost for trajectory smoothness:

[0018] This represents the second derivative of the candidate trajectory at time t. This represents the third derivative of the candidate trajectory at time t; L constraint Cost of violating anchor point constraints:

[0019] and These represent the starting pose and ending pose of the candidate trajectory, respectively. and These represent the starting pose and ending pose of the anchor point line, respectively. This represents the p-th necessary path point on the candidate trajectory. Indicates the anchor point line and The corresponding mandatory path point, where p represents the index of the mandatory path point. express and The distance between them; Energy cost of implementation: , This represents the trajectory state of the candidate trajectory at time t. Indicates a preset reference trajectory; The optimal trajectory is converted into a sequence of control commands that can be executed by the robot. The sequence of control commands includes joint spatial position / velocity / torque commands, end effector pose commands, and force control mode switching commands. The generated motion sequence is pre-validated in the robot simulation environment. If the validation fails, it is returned to resample and generate again. During execution, the deviation between the actual trajectory and the fitted trajectory is monitored in real time. When the deviation exceeds a preset threshold, the WASM large model is triggered to perform local online refitting only on the deviation segment.

[0020] Secondly, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the embodied intelligent robot motion generation method based on the WASM large model.

[0021] Compared with the prior art, the advantages of the present invention are as follows: 1. A revolutionary paradigm for experience storage and reuse; This invention abandons the traditional multi-layer decomposition paradigm of task → subtask → atomic skill → combined invocation, and uses the complete task execution trajectory line as the basic storage unit; More importantly, through representative screening and a constant library capacity mechanism, the size of the line library is strictly controlled within the constant range of task type × upper limit of each category, which fundamentally avoids the exponential computational complexity problem of massive primitive selection and permutation in the atomic skill library scheme. Even if the robot executes millions of tasks, the retrieval space is still maintained at the level of thousands or tens of thousands, and the retrieval latency is constant. 2. Line Fitting Alternative Skill Combinations Based on Conditional Diffusion Model. This invention utilizes a conditional diffusion model to fit multiple matching lines to generate new actions, rather than selecting and combining them one by one from an atomic skill library. Trajectory fitting maintains the temporal integrity and physical continuity of the action sequence, avoiding the problem of skill disconnection in combination methods. The conditional diffusion model, through a classifier-independent guidance strategy, can accurately satisfy anchor point constraints while ensuring the quality of generated trajectories. 3. Extremely low inference computational cost. This invention only calls the WASM large model in the line fitting stage, and uses matching lines as a strong prior condition, which greatly reduces the search space; for new tasks that are highly similar to historical tasks, existing lines can be directly reused through high similarity matching without model inference, realizing zero-computational-cost action generation; at the same time, the controllable size of the line library ensures that the computational cost of the retrieval stage is always at the constant level. 4. Natural adaptation of learning with few samples: This invention uses the anchor line mechanism to complete the representation of a new task in only 1 to 3 initial operations, and retrieves matches in the existing experience base based on the representation. This has obvious practical advantages compared with the existing methods that require a large amount of demonstration data. 5. A self-evolving and capacity-controlled experience accumulation mechanism. The representative line selection and dynamic simplification maintenance mechanism ensures that the line library is always kept concise and has a high information density. At the same time, each successfully executed new feature task is automatically converted into a representative line and added to the library, so that the robot's operational capabilities are continuously enhanced as the tasks become more diverse. 6. Prior reinforcement of statistical weights. Although redundant lines are not stored, their successful execution counts are accumulated through statistical weights, enabling the WASM large model to perceive which typical lines are the most common and reliable choices during fitting. This gives higher prior probabilities to high-frequency experiences, improving the reliability of generated trajectories.

[0022] Other beneficial effects of the present invention will be explained in detail through the introduction of specific technical features and technical solutions in specific embodiments. Those skilled in the art should be able to understand the beneficial technical effects brought about by these technical features and technical solutions through the introduction of these technical features and technical solutions. Attached Figure Description

[0023] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a schematic diagram of the overall technical process provided for an embodiment of the present invention; Figure 2 This is a schematic diagram of the line data structure provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the representative line screening and storage and storage capacity control process in step S1.4 of this invention. Figure 4 A schematic diagram of a similar line retrieval process based on key attribute matching provided in an embodiment of the present invention; Figure 5 A schematic diagram of the WASM large-model conditional diffusion model architecture (including forward diffusion and reverse denoising process) provided for an embodiment of the present invention. Figure 6 This is a schematic diagram of the WASM large model training process provided in an embodiment of the present invention. Detailed Implementation

[0024] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0025] This invention provides a method and device for generating embodied intelligent robot actions based on the WASM large model. It abandons the traditional paradigm of "breaking down tasks into atomic skills and then recombining them" and instead adopts a line library scheme that uses "complete task execution trajectories as basic storage units". Based on the conditional diffusion model, it realizes intelligent fitting and generalization generation of multiple experience lines to solve the problems of skill combination explosion, difficulty in explicit storage and reuse of experience, high inference computing power consumption, and difficulty in maintaining the integrity of action temporal sequence and physical continuity in the existing technology.

[0026] In this invention, a line refers to an experience storage unit based on the continuous key execution action data chain of a robot's complete task execution process; an anchor line refers to a line generated by the robot for a new task through a small number of initial operations, used as a query vector for matching in the line library; and the WASM (World Action State Model) large model refers to a large model of the world action state model that adopts the conditional diffusion model architecture.

[0027] Example 1: Reference Figure 1 As shown, this embodiment uses the operation task performed by a six-axis industrial robot on a precision assembly line as an example to illustrate the overall technical process. The method provided in this embodiment includes steps S1 to S6.

[0028] Step S1: Constructing the task execution trajectory line library; After each robot completes a real operation task, the system automatically collects multi-source data from the entire process, generates candidate lines, and stores them in a permanent line library after representative screening.

[0029] S1.1 Multi-source data acquisition: Acquire multi-source data generated by the robot during a complete task execution cycle, including: kinematic trajectory data, i.e., complete time series of joint angles, angular velocities, and angular accelerations; force data, i.e., continuous change curves of the magnitude and direction of contact forces / torques at each joint and contact forces at the end effector; visual state data, i.e., time series of images of the operation scene and the pose change trajectory of the manipulated object; task parameter data, i.e., task type identifier, geometric and physical properties of the manipulated object, and environmental constraint parameters; and temporal structure data, i.e., total task duration and time boundaries of each operation stage.

[0030] S1.2 Key Feature Extraction and Trajectory Compression: A multimodal fusion method is used to automatically identify the boundaries of key stages in the task execution process (based on force-sensory abrupt changes and visual state changes); key parametric features of each stage are extracted: displacement, rotation angle, velocity curve shape, and force control curve shape; lossy compression is performed on the original time series data to retain key feature points (extreme points, inflection points, and abrupt change points), and compression and reconstruction are achieved using an autoencoder based on time series Transformer.

[0031] S1.3 Line Structured Encapsulation: Encapsulate lines into a standardized line data structure (see reference). Figure 2 As shown, the line ID contains the following fields: Line ID, a globally unique identifier; Task type label, such as "bolt tightening" or "shaft hole assembly"; Object attribute vector, including object category, size, weight, material, surface characteristics, etc.; Scene attribute vector, including scene category, lighting conditions, spatial constraints, etc.; Key stage sequence, including type label, temporal boundary, and feature parameters for each stage; Compressed trajectory data, including compressed sequences of joint trajectories, force trajectories, and object state trajectories; Execution result label, which is success / failure / quality score; Statistical weight, which is the number of successful executions for the category represented by the line.

[0032] S1.4 Select and store representative lines (refer to) Figure 3 (As shown): The trajectory data generated by each task execution is first stored as candidate lines in a temporary buffer, and is not directly added to the permanent line library. The system executes the following library entry decision logic: 1) Similarity Clustering Judgment: Calculate the comprehensive feature similarity between the candidate line and all stored representative lines of the same task type in the line library, and evaluate it using the cosine similarity of the attribute feature vectors: Sim(t candidate , t stored ) = cos(f candidate , f stored ) Where, τ candidate Indicates candidate lines, τ stored f represents the representative lines that have been stored. candidate f represents the comprehensive attribute feature vector of the candidate lines. stored This represents the comprehensive attribute feature vector of the stored representative lines. Sim represents the comprehensive feature similarity; cos() represents the cosine similarity, which measures the directional similarity between two feature vectors in the feature space.

[0033] 2) Warehousing decision and storage capacity control: If Sim > θ high (θ)high If the similarity is between 0.92 and 0.98, it is considered high, and the candidate line is judged as a redundant expression of existing experience, rejected from the database, and the statistical weight w←w+1 of the representative line is updated to enhance its prior strength in subsequent fitting; if θ is satisfied... low <Sim≤θ high (θ) low If the similarity is between 0.65 and 0.80, it is considered to have moderate similarity. Candidate lines are then judged to contain new environmental features or operational fine-tuning, and are stored as new representative lines in the permanent line library. If Sim ≤ θ low If the similarity is low, it is considered a new task type and is directly added to the database as the first representative line of the new category.

[0034] 3) Maximum number of lines for a single task type: When the number of representative lines for a certain task type in the line library reaches the preset maximum number N... max (N) max When selecting 5 to 15 lines, the line merging mechanism is triggered: K-means clustering is performed on all representative lines of this task type (with N clusters). max / 2); after clustering, multiple lines within each category are merged into a single central line with stronger generalization; the trajectory data of the central line is the weighted average of the trajectories within the class, the attribute feature vector is the center within the class, and the statistical weights are accumulated; redundant individuals that are merged are deleted, keeping the library size no more than N. max .

[0035] 4) Index building: When the database is inserted, multi-dimensional indexes are built simultaneously, including classification indexes based on task type labels and vector indexes based on attribute feature vectors (such as FAISS or HNSW) to support efficient retrieval in step S3.

[0036] Step S2: Generate new task "anchor lines" based on a small number of real operations; When the robot encounters a new task, it performs 1 to 3 initial operations through any of the following methods: teaching operation, programming execution, video imitation learning, or remote operation. Data from the entire process is collected and encapsulated into "anchor line" according to steps S1.1 to S1.3, and the key attribute feature vectors of the task are extracted. f anchor As a query vector for subsequent matching.

[0037] Step S3: Similar line retrieval based on key attribute matching (refer to...) Figure 4 (as shown) With "anchor line" attribute feature vector f anchor To retrieve the vector, perform a multi-level search within the line library: S3.1 Coarse screening: Based on task type labels, the classification index is filtered to retrieve only the subset of lines with the same or similar task types, which greatly reduces the search space.

[0038] S3.2 Fine Screening: Based on the coarse screening results, vector similarity retrieval (cosine similarity) is performed based on the attribute feature vectors: Sim(f anchor , f i ) = f anchor ·f i / (‖f anchor ‖·‖f i ‖) S3.3 Comprehensive Scoring and Ranking: A multi-dimensional comprehensive score is applied to the candidate lines. Score = α·Sim attr + b·Sim scene + c·Sim structure + d·Quality in, Sim attr For attribute similarity, Sim scene For scene matching degree, Sim structure The stage structure matching degree is represented by Quality, which is the historical execution quality score; α+β+γ+δ=1, and each coefficient can be adaptively adjusted according to the task type.

[0039] S3.4 Output matching results: Select the K historical lines with the highest comprehensive scores as the matching results (K≥2). The value of K can be adaptively set according to the complexity of the task. The higher the complexity, the larger the value of K, and the maximum value shall not exceed N. max .

[0040] Step S4: Line fitting and motion prediction of the WASM large model based on the conditional diffusion model (refer to...) Figure 5 (as shown) S4.1 This step uses the K similar lines matched in step S3 as prior conditions, and performs line fitting based on the WASM large model with a conditional diffusion model architecture to generate the optimal motion trajectory suitable for the current task. The WASM large model includes four core components: trajectory encoder, conditional encoder, conditional diffusion model, and trajectory decoder.

[0041] (1) Track encoder: Based on the Transformer architecture, it encodes the time-series trajectory data of variable-length lines into a hidden state vector of fixed dimensions: z i = Enc traj (t i ),z i ∈ R d Where, τ i Let represent the trajectory sequence of the i-th input line. Enc traj Represents the trajectory encoding function. z i Let represent the hidden state vector of the trajectory of the i-th input line. R d Let represent a d-dimensional real vector space, where d represents the dimension of the latent space; (2) Conditional Encoder: This module fuses key attribute features and trajectory latent states into a comprehensive conditional vector, comprising three sub-modules. The prior distribution coding module extracts statistical priors from the K matching lines: c prior = Aggregate(z 1 , z 2 , …, z K ) The anchor point constraint encoding module encodes the key stage boundaries and key parameters in the anchor point lines into constraint vectors. c constraint This includes: starting pose, ending pose, necessary path points, force control curve envelope, and velocity limit; the fusion output module integrates prior, constraint, and anchor point attributes into a comprehensive condition vector. c = Fuse(c prior , c constraint , f anchor ) Here, Fuse represents the fusion function.

[0042] (3) Conditional Diffusion Model (Core Generation Module): Adopts the Denoising Diffusion Probability Model (DDPM) architecture. Gaussian noise is gradually added to the target trajectory τ0 during the forward diffusion process.

[0043] q ( :T|τ 0) =

[0044] in, The preset noise scheduling coefficients (using cosine scheduling) are used, and T is the number of diffusion steps (ranging from 50 to 200 steps). The reverse denoising process is guided by the conditional vector c, starting from pure noise... ~N(0, I) stepwise noise reduction:

[0045] The denoising network employs a Diffusion Transformer (DiT) architecture, taking trajectory sequence tokens and conditional vectors as input to predict added noise.

[0046] The denoising process employs a classifier-free guidance strategy to balance the quality of the generated trajectory with its condition compliance.

[0047] in, For guiding strength coefficient ( When the value is greater than 1, the influence of the condition is amplified; typically, the value is 2 to 8. This indicates an empty condition. The single-step update formula is:

[0048] in, =1 , , This represents the posterior variance.

[0049] (4) Trajectory decoder: Decodes the generated hidden state trajectory τ0 into a complete sequence of robot motion trajectories: = Dec traj (t 0 ) Dec traj (t 0 ) This indicates that the hidden state trajectory τ0 is being decoded; This is a sequence of robot motion trajectories. This includes joint angle trajectory, end-effector pose trajectory, and force control parameter trajectory.

[0050] S4.2 Candidate Trajectory Sampling and Optimization Selection: M candidate trajectories (M≥3) are sampled from the conditional distribution, and the optimal trajectory is selected based on the following objective function: t = argmin τ [l 1 ·L smooth (t) + λ 2 ·L constraint (t) + λ 3 ·L energy (τ)], τ ∈ {τ(1), τ(2), …, τ(M)} Where, τ This indicates the selected optimal trajectory. argmin τ This represents the trajectory selection operation that minimizes the objective function from the set of candidate trajectories. τ(1), τ(2), …, τ(M) Let λ1, λ2, and λ3 represent the candidate trajectories from the 1st to the Mth, where M represents the number of sampled candidate trajectories, and λ1, λ2, and λ3 are the weight coefficients of each cost. L smooth Cost for trajectory smoothness:

[0051] This represents the second derivative of the candidate trajectory at time t. This represents the third derivative of the candidate trajectory at time t; L constraint Cost of violating anchor point constraints:

[0052] and These represent the starting pose and ending pose of the candidate trajectory, respectively. and These represent the starting pose and ending pose of the anchor point line, respectively. This represents the p-th necessary path point on the candidate trajectory. Indicates the anchor point line and The corresponding mandatory path point, where p represents the index of the mandatory path point. express and The distance between them; Energy cost of implementation: , This represents the trajectory state of the candidate trajectory at time t. This indicates a preset reference trajectory.

[0053] S4.3 Key Stage Sequence Generation: In addition to continuous trajectory data, the WASM large model also outputs the fitted key stage sequence—the temporal boundaries and key parameters of each stage—for use in motion orchestration and monitoring in step S5.

[0054] WASM large model training method Reference Figure 6 As shown, the training of the WASM large model is divided into two stages.

[0055] Phase 1 is pre-training: The model weights are pre-trained on a large-scale dataset containing massive robot operation trajectories. A self-supervised learning approach is used, randomly masking or adding noise to the complete trajectory. The training model reconstructs the complete trajectory from partial observations, learning the prior distribution of the trajectory data. The pre-training loss function is:

[0056] Among them, expectations Taken from , , For large-scale pre-trained datasets, For added noise, The condition is empty.

[0057] Phase two is fine-tuning: first, initialization is performed, and then the line library dataset is fine-tuned. Conditional diffusion fine-tuning training is performed on the line library data accumulated by the robot to fine-tune the loss function; the conditional generation capability of the model is fine-tuned using historical lines in the line library as conditions.

[0058] Among them, expectations Taken from , For the line library dataset, This refers to several context lines (a set of condition lines) that belong to the same task type as the target trajectory τ. For the conditional encoding function, an auxiliary contrastive learning loss is further introduced to enhance the representational ability of the line encoder, enabling lines of the same task to cluster in the latent space:

[0059] in, For lines τ i The encoded vector, For lines τ j The encoded vector, The encoding vector for positive sample lines of the same task. For cosine similarity, Here, is the temperature coefficient (ranging from 0.1 to 0.5), and N is the number of lines participating in the comparison. The overall training loss is: L total = L ft + λcontrast ·L contrast in, l contrast To compare the weighting coefficients of the learning loss, To aid in the comparison of learning loss.

[0060] Step S5: Action generation and execution; S5.1 Motion Command Generation: The optimal trajectory τ generated by fitting the WASM large model. Converted into a sequence of control commands executable by the robot, including joint spatial position / velocity / torque commands, end effector pose commands, and force control mode switching commands.

[0061] S5.2 Simulation Pre-verification: Perform rapid pre-verification on the generated motion sequence in the robot simulation environment to check for kinematic conflicts, force control overruns, or collision risks; if the verification fails, return to step S4 to resample and generate.

[0062] S5.3 Actual Execution and Online Fine-tuning: The verified action commands are sent to the robot for execution; during the execution process, the deviation between the actual trajectory and the fitted trajectory is monitored in real time. When the deviation exceeds the preset threshold, the WASM large model is triggered to perform local online refitting (only the deviation segment is regenerated, rather than global replanning).

[0063] Step S6: Feedback on execution results and dynamic simplification of the line library; S6.1 Execution Result Evaluation: After the task is completed, the system automatically evaluates the execution effect, including task completion, operation accuracy, and execution efficiency.

[0064] S6.2 Conditional Incremental Update: Using the current execution trajectory as candidate lines, recall the representative lines with the highest comprehensive scores for this task type from the line library, and calculate the similarity between the two; high similarity (Sim > θ) high If the similarity is low or medium, the storage is abandoned and only the statistical weight w←w+1 of the representative line is updated; if the similarity is medium or low, the storage decision process of step S1.4 is executed.

[0065] S6.3 Dynamic Simplification and Maintenance of the Line Library: When the number of representative lines for a certain task type in the line library exceeds the preset upper limit N... max At that time, K-means clustering is triggered to merge multiple lines that are clustered into one class into a single central line, ensuring that the total size of the line library remains O(M·N) at any given time. max The system is designed to prevent the library from expanding indefinitely. (M represents the number of task types.)

[0066] Application Example: Shaft and Hole Assembly Tasks in Industrial Assembly Scenarios The implementation process is illustrated using a six-axis industrial robot performing shaft and hole assembly tasks on a precision assembly line as an example. Step S1: Initial accumulation of the line library: The robot successively performs shaft and hole assembly tasks of various specifications (diameter 10mm / 15mm / 20mm, material aluminum alloy / stainless steel, clearance fit / interference fit, etc.) on the production line. After each execution, the system generates candidate lines. Taking the 10mm diameter aluminum alloy clearance fit as an example, due to the high similarity of the candidate lines generated in the first few executions, only the first one is entered into the library as a representative line and its cumulative statistical weight w=3. Subsequent similar lines are not repeatedly entered into the library. Finally, only 8 representative lines (corresponding to 8 typical working condition combinations) are retained in the shaft and hole assembly category of the line library, with a total library capacity of approximately 200 lines.

[0067] New Task Scenario: The new task is to assemble a 12mm diameter copper shaft into a hole of the same material, with a transition fit. Step S2: The operator guides the robot to complete one initial operation via a teach pendant. The system generates "anchor point lines" and extracts key attributes: the object is a copper shaft, 12mm in diameter, made of copper, and the fit type is a transition fit. Step S3: The system uses attribute features as query vectors to search the line library and matches the 5 lines with the highest comprehensive scores, namely, a 10mm diameter copper shaft interference fit line, a 15mm diameter copper shaft transition fit line, a 10mm diameter aluminum alloy transition fit line, etc. Step S4: Input the 5 matching lines and anchor point conditions into the WASM conditional diffusion model. The condition encoder extracts the prior distribution features of the 5 lines (the commonality is the force control curve shape in the force control search stage, and the difference is the peak insertion force corresponding to different diameters), and merges them with the anchor point constraints (12mm diameter, copper, transition fit) into a condition vector. The diffusion model generates 20 candidate trajectories from random noise through 50 steps of denoising. After comprehensive evaluation of smoothness, constraint satisfaction, and energy consumption, the optimal trajectory is selected. Key generated results include the search amplitude in the force control search stage (based on 12mm diameter interpolation), the feed rate curve (based on transition fit optimization), and the expected peak insertion force. Step S5: The system converts the trajectory into control commands, verifies them through simulation, and then issues them for execution. The task is successfully completed. Step S6: The trajectory executed this time is used as a candidate line. The similarity Sim=0.78 with the 10mm diameter copper shaft interference fit lines in the library is at a medium level (including new diameter features). It is added to the library as a new representative line, increasing the number of shaft hole assembly category lines to 9 (not exceeding the upper limit N). max =10); thereafter, when encountering similar specification tasks, the line can be directly matched or reused with only minor adjustments.

[0068] Example 2 This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the intelligent generation method for embodied robot motion based on matching and fitting a large WASM model and a task execution trajectory line library as described in the above embodiment.

[0069] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, enabling the system and its various devices, modules, and units provided in this specific embodiment to be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided in this specific embodiment can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0070] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A method for generating embodied intelligent robot actions based on the WASM large model, characterized in that, include: Acquire multi-source data of the entire process of the robot's historical task execution, encapsulate the continuous key execution action data chain corresponding to each complete task execution process into candidate lines, perform representative screening on the candidate lines, and store the screened representative lines into a line library, with the complete task execution trajectory as the basic storage unit for the lines. When a new task is received, the entire process data is collected through at least one initial operation and encapsulated into anchor line. The attribute feature vector of the anchor line is then extracted as a query vector. The query vector is used to search the line library and match several similar lines. The matched similar lines are used as prior conditions and input into the WASM large model with a conditional diffusion model architecture for line fitting. The motion trajectory adapted to the current task is generated through the reverse denoising process. The motion trajectory is converted into a sequence of control instructions that can be executed by the robot and then executed. The representative screening includes: calculating the comprehensive feature similarity between the candidate lines and the representative lines already stored in the line library under the same task type. Sim(τ candidate , τ stored ) = cos(f candidate , f stored ) Where, τ candidate Indicates candidate lines, τ stored f represents the representative lines that have been stored. candidate f represents the comprehensive attribute feature vector of the candidate lines. stored This represents the comprehensive attribute feature vector of the stored representative lines. Sim represents the comprehensive feature similarity; cos() represents the cosine similarity, which measures the directional similarity between two feature vectors in the feature space. If Sim > θ high If a candidate line is determined to be a redundant expression based on existing experience, it is rejected from being added to the database, and the statistical weight w←w+1 of the representative line is updated. If θ low <Sim≤θ high If a candidate line is determined to contain new environmental features or operational fine-tuning, it is stored as a new representative line in the line library. If Sim≤θ low This is determined to be a new task type and is directly added to the database as the first representative line of the new category; among them, θ high For high similarity threshold, θ low For low similarity threshold, θ high The value range of θ is 0.92 to 0.

98. low The value range is 0.65 to 0.80, where w represents the number of successful executions for the category represented by the representative line; When the number of representative lines for a certain task type in the line library reaches the preset upper limit N max At this time, the line merging mechanism is triggered: Perform K-means clustering on all representative lines under this task type, with N clusters. max / 2, after clustering, multiple lines within each category are merged into a central line. The trajectory data of the central line is the weighted average of the trajectories within the category, the attribute feature vector is the center within the category, the statistical weights are accumulated, and the redundant individuals that are merged are deleted. Where, N max N is the preset upper limit for the number of representative lines under a single task type. max The value range is 5 to 15; When data is added to the database, a multi-dimensional index is constructed simultaneously. The multi-dimensional index includes a classification index based on task type labels and a vector index based on attribute feature vectors. After the task is completed, the execution trajectory is used as a representative candidate line for the selection process, thereby enabling dynamic updating and streamlined maintenance of the line library.

2. The method for generating embodied intelligent robot actions based on the WASM large model according to claim 1, characterized in that, The multi-source data includes: kinematic trajectory data, force data, visual state data, task parameter data, and temporal structure data; The kinematic trajectory data includes: a complete time series of joint angles, angular velocities, and angular accelerations; The force data includes: continuous curves showing the magnitude and direction of the contact force / torque at each joint and the contact force at the end effector; The visual state data includes: the temporal sequence of the operation scene image and the pose change trajectory of the operation object; The task parameter data includes: task type identifier, geometric and physical properties of the operation object, and environmental constraint parameters; The time-series structure data includes: the total duration of the task and the time boundaries of each operation stage; When encapsulating lines, a multimodal fusion method is used to identify the boundaries of key stages in the task execution process based on force abrupt changes and visual state changes, and to extract key parameterized features of each stage; The key parameterized features include displacement, rotation angle, velocity curve shape and force control curve shape. The original time series data is lossily compressed to retain key feature points in extreme points, inflection points and abrupt change points, and the lines are encapsulated into a standardized data structure. The data structure includes fields such as task type label, operation object attribute vector, scene attribute vector, key stage sequence, compressed trajectory data, execution result label, and statistical weight.

3. The method for generating embodied intelligent robot actions based on the WASM large model according to claim 1, characterized in that, The at least one initial operation is one to three initial operations performed through any one of the following methods: teaching operation, program execution, video imitation learning, or remote operation.

4. The method for generating embodied intelligent robot actions based on the WASM large model according to claim 1, characterized in that, The retrieval includes: a coarse screening based on task type labels for classification index filtering, retrieving only subsets of lines with the same or similar task types; and vector similarity retrieval based on attribute feature vectors on the coarse screening results. Sim(f anchor , f i ) = f anchor ·f i / (‖f anchor ‖·‖f i ‖) Then, the candidate lines are comprehensively scored from multiple dimensions: Score = α·Sim attr + β·Sim scene + γ·Sim structure + δ·Quality The K historical lines with the highest overall scores are selected as the matching results, where K ≥ 2; f anchor The attribute feature vector representing the anchor point line. f i Let represent the attribute feature vector of the i-th candidate line, where i represents the index of the candidate line, and ||·|| represents the magnitude of the vector. Sim attr Indicates attribute similarity. Sim scene Indicates scene matching degree. Sim structure Indicates the degree of matching of the stage structure. Quality The historical execution quality score is represented by α, β, γ, and δ, which are the score weight coefficients, and α+β+γ+δ=1. K represents the number of similar lines matched.

5. The method for generating embodied intelligent robot actions based on the WASM large model according to claim 4, characterized in that, The line fitting is achieved through the trajectory encoder, conditional encoder, conditional diffusion model, and trajectory decoder of the WASM large model; The trajectory encoder, based on the Transformer architecture, encodes variable-length line temporal trajectory data into a fixed-dimensional latent state vector: z i = Enc traj (τ i ),z i ∈ R d Where, τ i Let represent the trajectory sequence of the i-th input line. Enc traj Represents the trajectory encoding function. z i Let represent the hidden state vector of the trajectory of the i-th input line. R d Let represent a d-dimensional real vector space, where d represents the dimension of the latent space; The conditional encoder includes a priori distribution coding module, an anchor point constraint coding module, and a fusion output module; The prior distribution coding module extracts statistical priors from the K matching lines: c prior = Aggregate(z 1 , z 2 , …, z K ) The aggregation method employs mean pooling or weighted fusion based on cross-attention, with the weights of the weighted fusion and the overall score of each matched line being calculated. i Proportional; The anchor point constraint encoding module encodes the key stage boundaries and key parameters in the anchor point lines into constraint vectors. c constraint The constraint vector includes the starting pose, ending pose, necessary path points, force control curve envelope, and velocity limit; the fusion output module fuses the prior, constraint, and anchor point attributes into a comprehensive condition vector. c = Fuse(c prior , c constraint , f anchor ) in, c prior Represents the statistical prior vector. Aggregate Represents aggregate functions, z1, z2, ..., z K Score represents the hidden state vector of the trajectory of the first to the Kth matching lines. i This represents the overall score for the i-th matching line. c constraint Represents the constraint vector. f anchor The attribute feature vector representing the anchor point line. c This represents the combined condition vector obtained by fusion, and Fuse represents the fusion function.

6. The method for generating embodied intelligent robot actions based on the WASM large model according to claim 5, characterized in that, The conditional diffusion model adopts a denoised diffusion probability model architecture; Gaussian noise is gradually added to the target trajectory τ0 during the forward diffusion process: The reverse denoising process, guided by the comprehensive condition vector c, gradually denoises from pure noise to... : The denoising network employs a Diffusion Transformer (DiT) architecture, taking the trajectory sequence tokens and the comprehensive conditional vector as input to predict the added noise. Furthermore, the denoising process utilizes a classifier-independent guidance strategy. The single-step update formula is: in, Indicates the target trajectory. and Representing the t-th step and the t-th step respectively The noisy trajectory in step 1, where t represents the diffusion time step. Indicates by Forward noise addition The conditional probability distribution, where N represents a Gaussian distribution. Here, I represents the preset noise scheduling coefficients. Indicated by For conditions by Denoising The conditional probability distribution, and Let represent the mean and variance of the predictions made by the denoising network, respectively, and θ represent the parameters of the denoising network. This represents the noise predicted by the denoising network. Indicates an empty condition. For guiding strength coefficient, >1, This represents the noise prediction value after correction by classifier-independent guidance. =1 , t =α1α2…α t That is, the first t steps α s The product of these products, where s is the diffusion time step number. Let be the posterior variance, and z be random noise following a standard Gaussian distribution N(0, I); the diffusion step number T ranges from 50 to 200. The value range is 2 to 8.

7. The method for generating embodied intelligent robot actions based on the WASM large model according to claim 6, characterized in that, M candidate trajectories are sampled from the conditional distribution, where M ≥ 3. The optimal trajectory is selected based on the following objective function: τ = argmin τ [λ 1 ·L smooth (τ) + λ 2 ·L constraint (τ) + λ 3 ·L energy [(τ)],τ ∈ {τ(1), τ (2), …, τ(M)} Where, τ This indicates the selected optimal trajectory. argmin τ This represents the trajectory selection operation that minimizes the objective function among the candidate trajectories. τ(1), τ(2), …, τ(M) Let λ1, λ2, and λ3 represent the candidate trajectories from the 1st to the Mth, where M represents the number of sampled candidate trajectories, and λ1, λ2, and λ3 are the weight coefficients of each cost. L smooth Cost for trajectory smoothness: This represents the second derivative of the candidate trajectory at time t. This represents the third derivative of the candidate trajectory at time t; L constraint Cost of violating anchor point constraints: and These represent the starting pose and ending pose of the candidate trajectory, respectively. and These represent the starting pose and ending pose of the anchor point line, respectively. This represents the p-th necessary path point on the candidate trajectory. Indicates the anchor point line and The corresponding mandatory path point, where p represents the index of the mandatory path point. express and The distance between them; Energy cost of implementation: , This represents the trajectory state of the candidate trajectory at time t. Indicates a preset reference trajectory; The optimal trajectory is converted into a sequence of control commands that can be executed by the robot. The sequence of control commands includes joint spatial position / velocity / torque commands, end effector pose commands, and force control mode switching commands. The generated motion sequence is pre-validated in the robot simulation environment. If the validation fails, it is returned to resample and generate again. During execution, the deviation between the actual trajectory and the fitted trajectory is monitored in real time. When the deviation exceeds a preset threshold, the WASM large model is triggered to perform local online refitting only on the deviation segment.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is executed by the processor, it implements the steps of the embodied intelligent robot motion generation method based on the WASM large model as described in any one of claims 1 to 7.

Citation Information

Patent Citations

  • Method and apparatus for generating robot skills, and medium, cloud server and robot control system.

    WO2023273178A1

  • Industrial robot motion planning method based on diffusion model

    CN119217373A

  • Robot task guidance method and device based on large language model and medium

    CN120316192A