A stream diffusion robot operation strategy generation method based on maturity re-projection and expert routing

Abstract

This invention discloses a method for generating streaming diffusion robot operation strategies based on maturity reprojection and expert routing. The method first collects and encodes multimodal observations of the robot, then divides the motion trajectory into multiple planning segments, samples them, and adds noise during training. A causal intent encoder and a planning segment interaction module are built, and a maturity routing expert block is designed. Differential denoising is performed using expert networks to complete model training. During the inference phase, a streaming buffer is maintained. After denoising using DDIM, the front-end action tokens are executed. Maturity reprojection is performed on the clean trajectory after shifting the buffer to the left, and noise is added again to restore the homogeneous maturity structure within the planning segment. A cyclic rolling closed-loop control is then implemented. This invention eliminates inference bias caused by mixed maturity inputs through maturity reprojection, improves model robustness through mixed sampling, balances near-end fine control and far-end replanning, reduces closed-loop error accumulation, and improves the stability and success rate of fine operation tasks for robots such as robotic arms and dexterous hands.

Description

Technical Field

[0001] This invention belongs to the technical field of robot operation, specifically relating to a method for generating streaming diffusion robot operation strategies based on maturity reprojection and expert routing. Background Technology

[0002] Robotic manipulation tasks typically require robots to generate continuous action sequences capable of performing tasks such as grasping, placing, pushing, plugging, stacking, and opening / closing drawers, based on current visual observations, body state, and verbal commands. In recent years, imitation learning methods have used expert demonstration data to train robot control strategies, enabling robots to predict future actions under given observation conditions. Traditional deterministic regression strategies usually directly output a single action or a fixed sequence of actions. While these methods have fast inference speeds, they are prone to insufficient representation of action distribution in robotic tasks with multiple solutions, contact constraints, long temporal dependencies, and requirements for fine-grained local control. To improve the quality of action generation, diffusion strategies have been introduced into robot manipulation strategy generation. Diffusion strategies treat future action trajectories as a whole, obtaining action sequences by progressively denoising noisy trajectories, thus better modeling multimodal action distributions and long-term temporal consistency. However, standard diffusion strategies typically require multiple iterations of denoising, resulting in slow inference speeds and difficulty meeting the requirements of high-frequency closed-loop control in robots.

[0003] To improve inference efficiency, existing methods often employ action chunking or streaming generation mechanisms. This involves generating multiple future actions at once, re-observing the environment after executing a portion of the actions, and then generating the next action. Partial streaming diffusion methods further allocate different diffusion steps to actions in different time zones, placing near-field actions in a low-noise state for execution and far-field actions in a high-noise state for subsequent replanning. This mechanism provides a unified framework for precise near-field control and long-range far-field planning. However, in real-world closed-loop deployment, high-frequency control typically requires the robot to execute only the first few action tokens in the predicted action sequence, then shift the remaining action buffer to the left and continue planning based on the latest observations. This token-level partial execution approach has an inconsistency in granularity compared to the training phase's method of setting diffusion maturity in planning segments. Summary of the Invention

[0004] The main objective of this invention is to overcome the shortcomings and deficiencies of existing technologies and provide a method for generating streaming diffusion robot operation strategies based on maturity reprojection and expert routing. Through three core innovative mechanisms—hybrid maturity sampling training, maturity routing expert blocks, and maturity reprojection streaming updates—the method unifies and aligns offline training distribution with online closed-loop inference input distribution, simultaneously achieving precise near-end control and flexible far-end replanning, thereby improving the stability, continuity, and task success rate of robot's long-term fine-grained operations.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] In a first aspect, the present invention provides a method for generating operational strategies for a streaming diffusion robot based on maturity reprojection and expert routing, comprising the following steps:

[0007] S1. Obtain the robot's current observation information; the observation information includes environmental visual information, robot body state information, and task condition information.

[0008] S2. Perform multimodal conditional coding on the observation information and fuse it to obtain conditional features;

[0009] S3. Construct the future action trajectory and segment the future action trajectory into... Each planning segment contains [number] planning sections. An action token, satisfying , This represents the total length of the future motion trajectory.

[0010] S4. Assign maturity to each planning segment, and perform forward diffusion noise addition on the actions within each planning segment based on the assigned maturity.

[0011] S5. During the training phase, a hybrid maturity sampling mechanism is used to generate training maturity. The hybrid maturity sampling mechanism includes step sampling, perturbation step sampling, and global sampling.

[0012] S6. Construct a causal intent encoder to extract local action features of each planning segment, and then recursively aggregate historical action intents along the planning segment dimension;

[0013] S7. Construct a planning segment interaction module to perform self-attention modeling within the planning segment and cross-attention modeling of historical planning segments on noisy action tokens, while maintaining temporal causality.

[0014] S8. Construct a maturity-based routing expert block. Based on maturity embedding, planning segment location embedding, causal intent state, and observation summary, construct routing context features, calculate expert selection probability, and dynamically select an expert network to complete the corresponding planning segment denoising calculation.

[0015] S9. Construct a joint loss function that integrates action prediction loss, expert load balancing loss, and route stability loss, and train the maturity-aware diffusion strategy model.

[0016] S10. Initialize the streaming action buffer during the inference phase. The streaming action buffer is organized according to planning segments, and each planning segment corresponds to a maturity level.

[0017] S11. Use streaming DDIM to perform denoising calculations on the entire streaming action buffer and cache the clean action estimates generated in this round.

[0018] S12, retrieve the front end of the streaming motion buffer. An action token is sent to the robot controller for execution. After the robot completes its task, it re-collects environmental observations.

[0019] S13. Perform maturity reprojection operation: shift the cached clean action estimate to the left. Each token is used to construct a standard maturity ladder. Based on the standard maturity, the clean trajectory after the left shift is re-noised, so that the buffer is restored to a structure with homogeneous maturity within the planning segment.

[0020] S14. Update the streaming action buffer: Add a new action token to the end of the buffer and assign it the maximum maturity. ;

[0021] S15. Repeat steps S11 to S14 to perform closed-loop control until the task is completed or the maximum number of control steps is reached.

[0022] As a preferred technical solution, in step S1, the visual information includes multi-view images of different parts; the robot body state information includes joint angles, joint velocities, end effector pose, and gripper state; and the task condition information includes language instructions, task tags, or target descriptions.

[0023] As a preferred technical solution, step S2 specifically includes:

[0024] Language commands are input into a language encoder to obtain target semantic features, multi-view visual images are input into a visual encoding network to obtain visual features, and robot joints, end-effector poses, and gripper states are input into a state encoding network to obtain state features. The three types of features are then concatenated, cross-attention fusion is performed, or gating fusion is performed to obtain unified conditional features.

[0025] As a preferred technical solution, in step S4, the maturity value ranges from 0 to... , The maturity value represents the total number of diffusion steps. A smaller maturity value indicates lower noise and the action is closer to an executable state; a larger maturity value indicates lower noise in the current planning segment and is more suitable for long-term planning.

[0026] As a preferred technical solution, in step S5, during stepped sampling, the first... The maturity of each planning segment is sampled according to the following formula:

[0027] ;

[0028] In the perturbation ladder sampling, the following perturbation is added to the ladder maturity level:

[0029] ;

[0030] In global sampling, all planning segments share the same global maturity level, which is uniformly sampled from 0 to N-1.

[0031] As a preferred technical solution, in step S8, a maturity routing expert block is constructed, specifically as follows:

[0032] The routing context features are obtained by fusing maturity embedding, planning segment location embedding, mapped causal intent state, and mapped observation summary through a routing context encoder.

[0033] Input the routing context features into the routing network and output expert selection logits, then normalize them with Softmax to obtain the probabilities of each expert selection.

[0034] Based on the selection probability, one or more expert networks are selected to perform denoising feature extraction on the noisy actions of the current planning segment.

[0035] As a preferred technical solution, in step S9, the joint loss function is expressed as follows:

[0036]

[0037] in, Predict MSE loss for motion noise; This is the expert load balancing loss, used to promote the balanced activation of different experts and avoid the long-term inactivity of some experts. This is the stable loss of the routing logits, used to constrain the numerical stability of the routing logits; and These are the weighting coefficients for the corresponding loss terms.

[0038] As a preferred technical solution, in step S10, during initialization, each planning segment is initialized using Gaussian noise, and an initialization maturity ladder is set; the maturity of the front-end planning segment is low, and the maturity of the tail-end planning segment is high.

[0039] As a preferred technical solution, in step S13, the maturity reprojection includes:

[0040] Generate a specification maturity ladder based on linear interpolation of front-end and tail-end target maturity in the buffer zone;

[0041] Based on the diffusion coefficient corresponding to the standard maturity, Gaussian noise is reapplied to the clean action trajectory after left shift to eliminate the mixed maturity distribution caused by the left shift of the token and unify the maturity of all action tokens in each planning segment.

[0042] In a second aspect, the present invention provides a flow diffusion robot, the flow diffusion robot comprising:

[0043] At least one processor; and,

[0044] A memory communicatively connected to the at least one processor; wherein,

[0045] The memory stores computer program instructions that can be executed by the at least one processor, which enables the at least one processor to execute the streaming diffusion robot operation strategy generation method based on maturity reprojection and expert routing.

[0046] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0047] (1) This invention explicitly models the maturity of planning segments, enabling different time regions in the future action trajectory to have different diffusion states. Planning segments closer to the current time maintain low noise and are close to executable, while planning segments farther from the current time maintain higher noise and retain the ability to replan later. Thus, it simultaneously takes into account both precise control of near-end actions and long-term planning of far-end actions within the same generation framework. Through a hybrid maturity sampling mechanism, the model can be exposed to multiple combinations of planning segment maturity during the training phase, improving the robustness of the model in the face of non-standard maturity distributions during streaming closed-loop inference. Through maturity routing expert blocks, different expert paths can be dynamically selected for action regions corresponding to different maturity levels, improving the model's adaptability to both low-noise action refinement and high-noise long-term planning generation requirements. Through a causal intent encoder and a planning segment interaction module, long-term action intent transmission across planning segments is realized, while avoiding future information leakage.

[0048] (2) The present invention further restores the homogeneity of maturity within the planning segment after the execution of the token-level part through the maturity reprojection mechanism, thereby solving the granularity mismatch problem between the planning segment-level noise scheduling in the training phase and the execution of the token-level part in the inference phase; by using the clean action estimation of the cache to re-add noise, the inconsistency between the residual noise distribution and the target maturity ladder in the next round is reduced, thereby improving the stability of closed-loop inference; through the streaming buffer rolling update and partial action execution mechanism, the robot can reduce the computational overhead of generating a complete action sequence from zero in each round while maintaining the high-frequency closed-loop replanning capability, thereby improving the action continuity, robustness and task success rate in long-term, contact-sensitive and fine operation tasks. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a flowchart of a method for generating operation strategies for a streaming diffusion robot based on maturity reprojection and expert routing, according to an embodiment of the present invention.

[0051] Figure 2 This is a structural diagram of the flow diffusion robot according to an embodiment of the present invention. Detailed Implementation

[0052] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the scope of protection of the present application.

[0053] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0054] Definitions of abbreviations and key terms:

[0055] Maturity: This refers to the current diffusion step or noise level of the action planning segment, indicating how close the segment is to an executable action state. Lower maturity indicates less noise and the action is closer to being executable; higher maturity indicates more noise and the action is more suitable for remote planning and subsequent replanning.

[0056] Planning segment: A series of consecutive action segments obtained by dividing the future action trajectory according to time. Each planning segment contains multiple action tokens and shares a maturity level.

[0057] Action token: A single time step action in an action sequence.

[0058] Streaming buffer: A cache of future action trajectories continuously maintained during closed-loop inference, used to store the action state of the current time step and several future time steps.

[0059] Maturity reprojection: After executing some action tokens and shifting the streaming buffer to the left, the clean action estimates cached during the denoising process are used to re-noise the buffer, restoring it to a state of homogeneous maturity within the planned segment.

[0060] Maturity routing expert block: A neural network module that dynamically selects expert networks based on diffusion maturity, planning segment location, causal intention state, and observation summary.

[0061] DDIM: Denoising Diffusion Implicit Model, used to progressively derive low-noise or clean motion estimates from noisy motion trajectories.

[0062] This invention proposes a method for generating operational strategies for streaming diffusion robots based on maturity reprojection and expert routing. For example... Figure 1 As shown in the model framework diagram, this method divides the future action trajectory into multiple planning segments and explicitly defines the current diffusion step of each planning segment as maturity. During the training phase, mixed maturity sampling enhances the model's adaptability to different maturity combinations. In the model architecture, maturity routing expert blocks dynamically select expert networks for action regions of different maturity levels. During the inference phase, after the token-level execution, maturity reprojection restores the streaming buffer to a homogeneous canonical structure within the planning segment, thus resolving the inconsistency between planning granularity and execution granularity.

[0063] This invention can be deployed on robotic arms, dexterous hands, mobile manipulators, or other embodied intelligent robot platforms for maneuvering tasks such as grasping, placing, pushing, stacking, plugging, opening and closing drawers, and using tools under language or task-specific conditions. This method for generating streaming diffusion robot maneuvering strategies based on maturity reprojection and expert routing specifically includes the following steps:

[0064] S1: Obtain the robot's current observation information.

[0065] The robot acquires current observation information in each control cycle, including environmental visual information, robot body state information, and task condition information.

[0066] Furthermore, the visual information may include multi-view images such as head and wrist; the robot body state information may include joint angles, joint velocities, end effector pose, and gripper state; and the task condition information may include language commands, task tags, or target descriptions.

[0067] S2: Conditionally encode the observation information.

[0068] Furthermore, step S2 specifically involves:

[0069] The language instructions are input into the language encoder to obtain the target semantic features;

[0070] Visual observations are input into a visual encoding network to obtain visual features; the robot's body state is input into a state encoding network to obtain state features.

[0071] Subsequently, the modal features are concatenated, cross-attention fusion is performed, or gated fusion is used to obtain conditional features:

[0072] .

[0073] in, Indicates the conditional characteristics after fusion; Indicates the characteristics of language instructions or task objectives;

[0074] Indicates visual observation features; This represents the robot's physical state characteristics; Concat(·) represents the feature concatenation operation.

[0075] S3: Construct the future motion trajectory and planning segment structure.

[0076] Furthermore, step S3 specifically includes:

[0077] Let the length of the future motion trajectory be... Action dimension is The future trajectory of the action is then represented as:

[0078]

[0079] in, Indicates the current control moment; Indicates the first The action at each time step; This represents a sequence of future actions starting from the current moment. Indicates length is Action dimension is The real-valued action space.

[0080] Divide the future action trajectory into Each planning segment contains [number] planning sections. An action token that satisfies:

[0081]

[0082] The rearranged motion trajectory is represented as follows:

[0083]

[0084] in, For batch size, For the number of planned segments, The number of action tokens in each planning segment.

[0085] S4: Assign maturity to the planning segment and perform forward noise addition.

[0086] Furthermore, step S4 specifically involves:

[0087] Let the total number of steps in the diffusion process be... Assign maturity level to each planning segment:

[0088]

[0089] in, Indicates the first The diffusion step index for each planning segment; This represents the planning segment index, typically =0,...,h-1; a lower maturity level indicates that the current planning segment has lower noise and is closer to an executable action; a higher maturity level indicates that the current planning segment has higher noise and is more suitable for long-term planning. For planning segments The forward noise addition process for the action token in the code is as follows:

[0090]

[0091] in:

[0092]

[0093] Specifically, Indicates the first Within the planning segment Noisy actions for each token; This indicates the addition of Gaussian noise.

[0094] S5: Uses a hybrid maturity sampling mechanism to generate training maturity.

[0095] Furthermore, step S5 specifically involves:

[0096] The training phase employs a hybrid maturity sampling strategy, including step sampling, perturbation step sampling, and global sampling.

[0097] In stepped sampling, the first The maturity level of each planning segment is determined using the following formula:

[0098]

[0099] In the perturbation ladder sampling, a perturbation is added to the ladder maturity level:

[0100]

[0101] in: In global sampling, all planning segments share the same maturity level:

[0102]

[0103] in, This indicates that the maturity level is shared globally; it means that all planning segments use the same maturity level during global sampling.

[0104] S6: Construct a causal intent encoder.

[0105] Furthermore, step S6 specifically involves:

[0106] For each planning segment, local action features are first extracted:

[0107]

[0108] in, Indicates the first Local motion characteristics of each planning segment.

[0109] Then, the historical action intent is recursively aggregated along the planning segment dimension:

[0110]

[0111] in, Indicates the first The causal intention state corresponding to each planning segment. This indicates a gated loop unit.

[0112] S7: Construct the planning segment interaction module.

[0113] Furthermore, step S7 specifically includes:

[0114] The noisy action token and the causal intention state are input into the planning segment interaction module. First, self-attention modeling is performed within the planning segment, and then cross-attention modeling is performed on the intention state of the historical planning segment, so that the current planning segment can utilize historical action trend information while maintaining temporal causality.

[0115] S8: Build maturity routing expert blocks.

[0116] Furthermore, step S8 specifically includes:

[0117] For the For each planning segment, the routing context features are first constructed, including maturity embedding, planning segment location embedding, causal intent state, and observation summary features:

[0118]

[0119] in, Indicates the route context encoder. Indicates the first The routing context features of each planning segment are then analyzed. The expert selection probability is then calculated.

[0120]

[0121] Based on the probability, one or more expert networks are dynamically selected to participate in the denoising calculation of the current planning segment.

[0122] S9: Training maturity awareness diffusion strategy.

[0123] Furthermore, step S9 specifically includes:

[0124] The model predicts the noise corresponding to each action token:

[0125]

[0126] in, This is the maturity matrix for the planning segment. The training objective is:

[0127]

[0128] The overall loss can be further included in the expert load balancing loss and the routing stability loss:

[0129]

[0130] in, This is used to promote the balanced activation of different experts and avoid the long-term inactivation of some experts. Used to constrain the numerical stability of routing logits; and These are the weighting coefficients for the corresponding loss terms.

[0131] S10: Initialize the streaming action buffer.

[0132] Furthermore, step S10 specifically includes:

[0133] Maintaining a streaming action buffer during the inference phase: The streaming action buffer is organized by planning segments, with each segment corresponding to a maturity level. During initialization, each planning segment is initialized using Gaussian noise, and an initial maturity ladder is set.

[0134]

[0135] Among them, the planning segments near the front of the buffer zone have lower maturity, while the planning segments near the rear have higher maturity.

[0136] S11: Perform streaming DDIM denoising.

[0137] Furthermore, step S11 specifically includes:

[0138] The model predicts noise across the entire streaming action buffer and updates the current clean action estimate using DDIM: Then update the action status for the next moment: in, This is the noise reduction step size. The system caches the estimated clean actions generated in the current round. This is used for subsequent maturity reprojection.

[0139] S12: Execute part of the action token.

[0140] Furthermore, step S12 specifically includes:

[0141] Take the previous section from the front planning segment of the streaming action buffer. Each action token is input into the robot controller for execution, where: After execution, the robot reacquires the current environmental observations and enters the next round of closed-loop reasoning.

[0142] S13: Perform maturity reprojection.

[0143] Furthermore, step S13 specifically includes:

[0144] After executing a partial action token, estimate the cached clean actions. Shift left Using one token, we obtain the clean trajectory after left shift: Then, a maturity ladder for the specifications is constructed: in, Indicates the maturity level of the target at the front end of the buffer zone. This indicates the maturity level of the target at the tail end of the buffer zone.

[0145] Re-add noise based on the target maturity level:

[0146]

[0147] in: Indicates the number of reprojections after the first projection. The first planning section A new noisy action for each token; This represents the clean motion estimate after a left shift. By reprojecting maturity, the left-shifted flow action buffer is restored to a homogeneous maturity structure within the planning segment.

[0148] S14: Update the streaming action buffer.

[0149] Furthermore, step S5 specifically involves:

[0150] After completing the maturity reprojection, reset the newly added action tokens at the end of the streaming action buffer to Gaussian noise and assign them the maximum maturity: The updated streaming action buffer serves as the input for the next round of closed-loop inference.

[0151] S15: Execute rolling closed-loop control.

[0152] Furthermore, step S5 specifically involves:

[0153] The system combines the latest observations of the robot with repeated execution steps S11 to S14 to achieve continuous streaming generation, partial execution and online replanning of the robot's motion trajectory until the task is completed or the maximum number of control steps is reached.

[0154] This invention addresses the core problem of mismatch between the granularity of training planning segments and the execution granularity of inference tokens through a maturity reprojection mechanism. After executing some actions and shifting the buffer to the left, the clean action estimate from the current round is used to re-noise to the standard step maturity level, restoring the homogeneous maturity structure within each planning segment. This eliminates the inference distribution shift caused by abnormal mixed maturity inputs, suppresses the accumulation of multi-round closed-loop errors, and significantly improves the stability of robot actions and the continuity of trajectory.

[0155] In another embodiment of the present invention, a flow diffusion robot is provided. Figure 2 Only a flow diffusion robot with components is shown; those skilled in the art will understand that... Figure 2The structure shown does not constitute a limitation on the flow diffusion robot and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0156] The streaming diffusion robot's operation strategy generation program, based on maturity reprojection and expert routing, stored in the first memory, is a combination of multiple instructions. When run in the first processor, it can achieve the following:

[0157] S1. Obtain the robot's current observation information; the observation information includes environmental visual information, robot body state information, and task condition information.

[0158] S2. Perform multimodal conditional coding on the observation information and fuse it to obtain conditional features;

[0159] S3. Construct the future action trajectory and segment the future action trajectory into... Each planning segment contains [number] planning sections. An action token, satisfying , This represents the total length of the future motion trajectory.

[0160] S4. Assign maturity to each planning segment, and perform forward diffusion noise addition on the actions within each planning segment based on the assigned maturity.

[0161] S5. During the training phase, a hybrid maturity sampling mechanism is used to generate training maturity. The hybrid maturity sampling mechanism includes step sampling, perturbation step sampling, and global sampling.

[0162] S6. Construct a causal intent encoder to extract local action features of each planning segment, and then recursively aggregate historical action intents along the planning segment dimension;

[0163] S7. Construct a planning segment interaction module to perform self-attention modeling within the planning segment and cross-attention modeling of historical planning segments on noisy action tokens, while maintaining temporal causality.

[0164] S8. Construct a maturity-based routing expert block. Based on maturity embedding, planning segment location embedding, causal intent state, and observation summary, construct routing context features, calculate expert selection probability, and dynamically select an expert network to complete the corresponding planning segment denoising calculation.

[0165] S9. Construct a joint loss function that integrates action prediction loss, expert load balancing loss, and route stability loss, and train the maturity-aware diffusion strategy model.

[0166] S10. Initialize the streaming action buffer during the inference phase. The streaming action buffer is organized according to planning segments, and each planning segment corresponds to a maturity level.

[0167] S11. Use streaming DDIM to perform denoising calculations on the entire streaming action buffer and cache the clean action estimates generated in this round.

[0168] S12, retrieve the front end of the streaming motion buffer. An action token is sent to the robot controller for execution. After the robot completes its task, it re-collects environmental observations.

[0169] S13. Perform maturity reprojection operation: shift the cached clean action estimate to the left. Each token is used to construct a standard maturity ladder. Based on the standard maturity, the clean trajectory after the left shift is re-noised, so that the buffer is restored to a structure with homogeneous maturity within the planning segment.

[0170] S14. Update the streaming action buffer: Add a new action token to the end of the buffer and assign it the maximum maturity. ;

[0171] S15. Repeat steps S11 to S14 to perform closed-loop control until the task is completed or the maximum number of control steps is reached.

[0172] Furthermore, if the modules / units integrated into the flow diffusion robot are implemented as software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0173] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0174] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0175] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for generating operational strategies for streaming diffusion robots based on maturity reprojection and expert routing, characterized in that, Includes the following steps: S1. Obtain the robot's current observation information; the observation information includes environmental visual information, robot body state information, and task condition information. S2. Perform multimodal conditional coding on the observation information and fuse it to obtain conditional features; S3. Construct the future action trajectory and segment the future action trajectory into... Each planning segment contains [number] planning sections. An action token, satisfying , This represents the total length of the future motion trajectory. S4. Assign maturity to each planning segment, and perform forward diffusion noise addition on the actions within each planning segment based on the assigned maturity. S5. During the training phase, a hybrid maturity sampling mechanism is used to generate training maturity. The hybrid maturity sampling mechanism includes step sampling, perturbation step sampling, and global sampling. S6. Construct a causal intent encoder to extract local action features of each planning segment, and then recursively aggregate historical action intents along the planning segment dimension; S7. Construct a planning segment interaction module to perform self-attention modeling within the planning segment and cross-attention modeling of historical planning segments on noisy action tokens, while maintaining temporal causality. S8. Construct a maturity-based routing expert block. Based on maturity embedding, planning segment location embedding, causal intent state, and observation summary, construct routing context features, calculate expert selection probability, and dynamically select an expert network to complete the corresponding planning segment denoising calculation. S9. Construct a joint loss function that integrates action prediction loss, expert load balancing loss, and route stability loss, and train the maturity-aware diffusion strategy model. S10. Initialize the streaming action buffer during the inference phase. The streaming action buffer is organized according to planning segments, and each planning segment corresponds to a maturity level. S11. Use streaming DDIM to perform denoising calculations on the entire streaming action buffer and cache the clean action estimates generated in this round. S12, retrieve the front end of the streaming motion buffer. An action token is sent to the robot controller for execution. After the robot completes its task, it re-collects environmental observations. S13. Perform maturity reprojection operation: shift the cached clean action estimate to the left. Each token is used to construct a standard maturity ladder. Based on the standard maturity, the clean trajectory after the left shift is re-noised, so that the buffer is restored to a structure with homogeneous maturity within the planning segment. S14. Update the streaming action buffer: Add a new action token to the end of the buffer and assign it the maximum maturity. ; S15. Repeat steps S11 to S14 to perform closed-loop control until the task is completed or the maximum number of control steps is reached.

2. The method for generating streaming diffusion robot operation strategies based on maturity reprojection and expert routing according to claim 1, characterized in that, In step S1, the visual information includes multi-view images of different parts; the robot body state information includes joint angles, joint velocities, end effector pose, and gripper state; and the task condition information includes language commands, task tags, or target descriptions.

3. The method for generating streaming diffusion robot operation strategies based on maturity reprojection and expert routing according to claim 1, characterized in that, Step S2 is as follows: Language commands are input into a language encoder to obtain target semantic features, multi-view visual images are input into a visual encoding network to obtain visual features, and robot joints, end poses, and gripper states are input into a state encoding network to obtain state features. By splicing, cross-attention fusion, or gating fusion of the three types of features, a unified conditional feature is obtained.

4. The method for generating streaming diffusion robot operation strategies based on maturity reprojection and expert routing according to claim 1, characterized in that, In step S4, the maturity value ranges from 0 to... , The maturity value represents the total number of diffusion steps. A smaller maturity value indicates lower noise and the action is closer to an executable state; a larger maturity value indicates lower noise in the current planning segment and is more suitable for long-term planning.

5. The method for generating streaming diffusion robot operation strategies based on maturity reprojection and expert routing according to claim 1, characterized in that, In step S5, during stepped sampling, the first... The maturity of each planning segment is sampled according to the following formula: ； In the perturbation ladder sampling, the following perturbation is added to the ladder maturity level: ； In global sampling, all planning segments share the same global maturity level, which is uniformly sampled from 0 to N-1.

6. The method for generating streaming diffusion robot operation strategies based on maturity reprojection and expert routing according to claim 1, characterized in that, In step S8, the maturity routing expert block is constructed, specifically as follows: The routing context features are obtained by fusing maturity embedding, planning segment location embedding, mapped causal intent state, and mapped observation summary through a routing context encoder. Input the routing context features into the routing network and output expert selection logits, then normalize them with Softmax to obtain the probabilities of each expert selection. Based on the selection probability, one or more expert networks are selected to perform denoising feature extraction on the noisy actions of the current planning segment.

7. The method for generating streaming diffusion robot operation strategies based on maturity reprojection and expert routing according to claim 1, characterized in that, In step S9, the joint loss function is expressed as follows: in, Predict MSE loss for motion noise; This is the expert load balancing loss, used to promote the balanced activation of different experts and avoid the long-term inactivity of some experts. This is the stability loss of the routing logits, used to constrain the numerical stability of the routing logits; and These are the weighting coefficients for the corresponding loss terms.

8. The method for generating streaming diffusion robot operation strategies based on maturity reprojection and expert routing according to claim 1, characterized in that, In step S10, during initialization, each planning segment is initialized using Gaussian noise, and an initialization maturity ladder is set; the front-end planning segment has low maturity, and the tail-end planning segment has high maturity.

9. The method for generating streaming diffusion robot operation strategies based on maturity reprojection and expert routing according to claim 1, characterized in that, In step S13, the maturity reprojection includes: Generate a specification maturity ladder based on linear interpolation of front-end and tail-end target maturity in the buffer zone; Based on the diffusion coefficient corresponding to the standard maturity, Gaussian noise is reapplied to the clean action trajectory after left shift to eliminate the mixed maturity distribution caused by the left shift of the token and unify the maturity of all action tokens in each planning segment.

10. A flow diffusion robot, characterized in that, The flow diffusion robot includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores computer program instructions that can be executed by the at least one processor, which are executed by the at least one processor to enable the at least one processor to execute the streaming diffusion robot operation strategy generation method based on maturity reprojection and expert routing as described in any one of claims 1-9.

Images