Embodied agent training method for generating 3D perception and thinking chain based on flow matching
By combining flow matching generation and thought chain, the problem of non-smoothness and robustness of action generation of embodied intelligent agents in 3D environment is solved, and the stable and efficient execution of embodied intelligent agents in complex environment is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHEJIANG UNIV OF TECH
- Filing Date
- 2026-03-12
- Publication Date
- 2026-06-05
AI Technical Summary
Existing methods for generating actions in embodied intelligent agents in 3D environments suffer from problems such as unsmooth trajectories, unnatural transitions, lack of explicit modeling of 3D structures, chaotic decomposition of long-term tasks, and insufficient robustness against disturbances.
By combining flow matching generation method with 3D perception and thought chain, multiple semantic heterogeneous reasoning paths are generated to verify potential spatial perturbations and screen action trajectory stability. The optimal action sequence is selected by calculating consistency score using deterministic vector field and performing post-event semantic verification to achieve self-correction.
It enhances the embodied intelligent agent's three-dimensional perception capabilities, task planning diversity, and physical feasibility of actions in uncertain environments, ensuring robust execution of long-term tasks.
Smart Images
Figure CN122154811A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence, specifically to a method for training embodied intelligent agents based on flow matching to generate 3D perception and thought chains. Background Technology
[0002] In the field of Vision-Language-Motion (VLA) technology, embodied agents, as intelligent systems with environmental perception and physical interaction capabilities, typically acquire environmental observation data through cameras and various sensors, combine this data with language commands for understanding and parsing, and ultimately generate executable action sequences to complete multi-step tasks. To enable agents to generate actions based on commands, existing research largely relies on large-scale teaching data for training, allowing the model to progressively infer and generate actions under multimodal input conditions. For example, the RT-1 method, based on 130,000 pieces of world interaction demonstration data, achieves visual and text fusion through feature modulation mechanisms; the Octo method constructs a rewardless imitation learning strategy using 1.5 million pieces of cross-device demonstration data, enabling the model to generalize across different robotic platforms. However, these methods generally employ discrete action sequences or progressive action regression, which may still result in problems such as uneven trajectories and unnatural transitions when the robotic arm performs in a real environment.
[0003] To improve the continuity and physical feasibility of motion, researchers have proposed motion modeling methods based on diffusion generation. These methods treat motion generation as a conditional denoising process, generating continuous and diverse motion trajectories through a sampling process. Representative works include Diffusion-VLA and CogACT. Although diffusion models can generate smoother motion trajectories, their reverse process relies on noise scheduling and multi-step iterations, resulting in slow inference speeds and difficulty in providing interpretable directions of dynamic changes.
[0004] To overcome the aforementioned limitations, flow matching has been introduced into the field of action generation. This method falls under the category of diffusion, learning a deterministic vector field to give directional characteristics to trajectory evolution in the latent space, thereby achieving faster and more stable action generation. Compared to traditional diffusion models, flow matching offers advantages such as shorter computational paths, higher sampling efficiency, and more stable convergence behavior.
[0005] Despite significant progress in flow matching methods for generating motion trajectories, several limitations remain: First, traditional diffusion methods rely heavily on 2D visual features or planar spatial representations, lacking explicit modeling of the scene's 3D structure. This leads to inaccurate target localization and mismatches between generated trajectories and real geometry. Second, in long-duration or multi-subtask reasoning, existing models typically employ single-step prediction strategies, resulting in chaotic task decomposition, a lack of logical connections between action sequences, and the accumulation of errors over long sequences, making it difficult to accurately correspond sub-actions to linguistic semantics. Third, existing models often blindly output sampling results, lacking consideration for the consistency of flow field velocity. While the generated trajectories may be geometrically continuous, their smoothness and interpretability during actual action execution are insufficient. Fourth, when faced with minor input perturbations, existing models lack inherent adversarial robustness. The generated strategies often exhibit significant instability, lacking a self-consistent verification mechanism based on perturbation consistency, making it difficult to fundamentally guarantee the robot's inherent safety in uncertain environments. Summary of the Invention
[0006] To address the problems existing in the prior art, this invention proposes an embodied intelligent agent training method based on flow matching to generate 3D perception and thought chain. By integrating 3D perception, multi-path reasoning and physical verification, it generates multiple semantically heterogeneous reasoning paths. Even under noise and occlusion, it can accurately recover the environmental structure, ensure the physical feasibility and stability of actions, and achieve self-correction and dynamic backtracking, ensuring the stable and safe operation of the embodied intelligent agent.
[0007] To achieve the above-mentioned objectives, this invention provides a method for training embodied intelligent agents based on flow matching to generate 3D perception and thought chains, comprising the following steps: Step 1: Receive visual images and language instructions, and generate a three-dimensional geometric structure perception representation in the latent space through a pre-trained flow matching model; Step 2: Based on language instructions and three-dimensional geometric structure perception representation, use an autoregressive thought chain generator to generate multiple semantically heterogeneous reasoning paths, and parse each reasoning path into a structured semantic strategy. Then, fuse the structured semantic strategy with the three-dimensional geometric structure perception representation to obtain multiple semantically heterogeneous conditions. Step 3: Input the semantic heterogeneous conditions into the flow matching model to guide the generation of continuous action trajectories that match the structured semantic strategy, and then obtain candidate trajectories through kinematic screening; Step 4: For each candidate trajectory, apply perturbation to the corresponding semantic heterogeneous conditions to guide the flow matching model to generate a verification trajectory. Based on the action trajectory and the verification trajectory, perform stability verification and screening to obtain a set of safe trajectories. Step 5: Using the deterministic vector field learned by the flow matching model as a benchmark, calculate the consistency score between the actual motion direction of each trajectory in the safe trajectory set at each state point and the predicted direction of the deterministic vector field, and select the optimal action sequence based on the consistency score. Step 6: Execute the optimal action sequence and use the autoregressive thought chain generator to perform semantic verification on the execution result. If the verification fails, select the suboptimal trajectory from the safe trajectory set and re-execute it to realize the training and dynamic backtracking of the embodied agent.
[0008] Preferably, the training process of the flow matching model in step 1 includes: A training sample set consisting of visual images, language instructions, and corresponding real 3D annotations is obtained. The visual images and language instructions in each training sample are extracted into visual features and language features respectively by the encoder, and then concatenated to form the original joint conditions. Construct a linear interpolation path from standard Gaussian noise distribution to real 3D annotation; In each training iteration, the original joint conditions are masked or injected with noise in conjunction with an anti-interference mechanism to establish impaired joint conditions. Using the instantaneous velocity of the linear interpolation path as the optimization objective, a flow matching model loss function is established to optimize the parameters of the flow matching model, resulting in the optimized flow matching model.
[0009] More preferably, the loss function of the flow matching model is expressed as: , In the formula, The loss function for the flow matching model is... For the parameters of the flow matching model, For time steps, Standard Gaussian noise, For the perceived representation of the target's three-dimensional geometric structure. For the damaged joint conditions, From arrive The linear interpolation path, For flow matching model, For mathematical expectation, It is the square of the L2 norm.
[0010] Preferably, in step 2, the generation of multiple semantically heterogeneous reasoning paths using an autoregressive thought chain generator includes: Using a pre-trained large language model as an autoregressive thought chain generator, with language instructions and three-dimensional geometric structure perception representation as conditional inputs, a high-temperature sampling strategy is adopted to perform multiple independent autoregressive generation, resulting in a set of multiple semantically heterogeneous reasoning path sequences, where each reasoning path sequence consists of a series of discrete subtask tokens.
[0011] Preferably, in step 2, the process of parsing each reasoning path into a structured semantic strategy includes: For each inference path sequence, a generator function is described in fine granularity. Parsing into a structured semantic strategy description: ,in Indicates the order of operations. Represents an object dependency graph. Represents spatial action constraints. Index for the reasoning path sequence; A text encoder is used to encode structured semantic policy descriptions into a sequence of policy embedding vectors.
[0012] By explicitly computing the high-level task logic (operation sequence and object dependency graph) of the embodied agent with the low-level geometric constraints (spatial action constraints) through a structured semantic strategy, which is the key interface connecting the "thought chain (cognitive layer)" and the "flow matching vector field (physical layer)," it ensures that the generated diverse action trajectories are not only geometrically reasonable but also logically correct. Furthermore, domain personnel can understand and debug the decision-making logic of the embodied agent by checking the structured semantic strategy, or impose artificial priority constraints by modifying the structured semantic strategy to achieve training.
[0013] Preferably, in step 2, the fusion of multiple semantic heterogeneous conditions includes: using the three-dimensional geometric structure perception representation as the query, using the strategy embedding vector sequence corresponding to the reasoning path as the key and value, and calculating the semantically injected three-dimensional geometric structure perception representation through a cross-modal attention mechanism; The semantically injected 3D geometric structure perception representation is fused with the original 3D geometric structure perception representation to form a semantically heterogeneous condition.
[0014] Preferably, step 3 specifically includes: For each semantically heterogeneous condition, the flow matching model defines a conditionally dependent deterministic velocity field; by sampling the initial action state from a standard Gaussian distribution and using a numerical ordinary differential equation solver to numerically integrate the deterministic velocity field in the time domain, a continuous action trajectory matching the structured semantic strategy is generated. Define a kinematic constraint function for the embodied intelligent agent. This kinematic constraint function includes joint angle restrictions, workspace boundaries, and singularity detection. The action trajectory is filtered by the kinematic constraint function to obtain candidate trajectories. The kinematic constraint function is expressed as: , In the formula, For the trajectory of the action The kinematic constraint functions, For time steps The trajectory of movement, This is the minimum limit for the joint angle. This is the maximum limit for the joint angle. This represents the workspace boundary.
[0015] Based on the deterministic velocity field, the low-pass filtering characteristics of the ordinary differential equation solver automatically smooth out potential high-frequency control noise during numerical integration and trajectory smoothing, effectively suppressing the slight jitter during the execution of the robotic arm. Furthermore, kinematic constraint functions are established to eliminate "degenerate solutions" that obviously violate physical constraints or are unreachable, ensuring that computational resources are used only to evaluate physically feasible trajectories.
[0016] Preferably, the process of generating the verification trajectory in step 4 includes: For each action trajectory and its corresponding semantic heterogeneous conditions, multiple sets of random perturbation vectors or random mask matrices are generated as perturbations. These perturbations are then injected into the original semantic heterogeneous conditions to obtain a set of stress test conditions. The random perturbation vectors follow a Gaussian distribution. Each condition in the set of stress test conditions is input into the flow matching model in parallel; the flow matching model generates the corresponding verification trajectory through an ordinary differential equation solver based on the same initial noise state, which is used for subsequent stability verification.
[0017] Preferably, in step 4, stability verification and filtering are performed based on the action trajectory and verification trajectory to obtain a set of safe trajectories, including: The perturbation consistency index between candidate trajectories and all verified trajectories is calculated. Combining three-dimensional geometric structure perception representation and structured semantic strategy, a safety threshold is dynamically set. If the perturbation consistency index of a candidate trajectory is lower than the threshold, it is determined to have intrinsic safety and is retained; otherwise, it is discarded, thus obtaining a set of safe trajectories. The disturbance consistency index is expressed as: , Indicates the number of candidate trajectory samples. Indicates the first Candidate trajectories, No. The verification trajectory corresponding to each candidate trajectory. This indicates the index of the verification trajectory sample. This represents the square of the L2 norm.
[0018] More preferably, the safety threshold is 0.01-0.05.
[0019] Preferably, the training process in step 5, which uses the deterministic vector field learned by the flow matching model as a benchmark, includes: Based on the distribution of expert data, an interpolation path from standard Gaussian noise to the target optimal action sequence is constructed for each training sample, and the instantaneous velocity of the interpolation path at each state point is defined as the target vector field. The flow matching model is trained by optimizing its parameters by minimizing the following conditional regression loss function, which is used to simulate the optimal action evolution implicit in the expert data and conforms to physical laws and task specifications: , in, For conditional regression loss, For flow matching model, For the parameters of the flow matching model, For time steps, For mathematical expectation, For expert data distribution, For the target condition state, For time steps The interpolation state, Let this be the target vector field, used to predict the state from the current state. Correctly evolve to the target The ideal direction It is the square of the L2 norm.
[0020] Preferably, in step 5, selecting the optimal action sequence based on the consistency score includes: Discretize the safety trajectories in the set of safety trajectories into a time series; At each time step of the time series, the actual tangent direction of the safety trajectory is calculated, and the predicted vector field direction of the flow matching model at the current state point is obtained. Calculate the cosine similarity between the actual tangent direction and the predicted vector field direction, and aggregate the cosine similarities across all time steps to obtain a consistency score: , In the formula, For the first Consistency score of each safety trajectory The fit of the internalized expert teaching distribution in the flow matching model was quantified; The total number of time steps. For safe trajectory in time step The actual tangent direction, For the flow matching model at time step and the current state point The direction of the predicted vector field at that location. It is an L1 norm. It is a smoothing constant; The safe trajectories in the set of safe trajectories are sorted according to their consistency scores, and the sequence with the highest consistency score is selected as the optimal action sequence.
[0021] Preferably, in step 6, the semantic verification of the execution result using the autoregressive thought chain generator includes: The real images collected after executing the optimal action sequence are encoded as descriptive information and combined with the text target of the current embodied agent training task to form a verification prompt; The verification prompt is input into the autoregressive thought chain generator, which switches to the discriminative mode and outputs the conditional probability of the completion of the current embodied agent training task through the classification layer or directly generates the judgment result to complete the semantic verification.
[0022] Compared with the prior art, the beneficial effects of the present invention include at least the following: This invention discloses a training method for embodied agents based on flow matching and thought chains. It generates a 3D geometric structure perception representation from multimodal inputs using a flow matching model trained to withstand interference. Based on this 3D geometric structure perception representation, a thought chain generator employing a high-temperature sampling strategy generates multiple semantically heterogeneous inference paths and parses them into structured semantic policies. The structured semantic policies are fused with the 3D geometric structure perception representation as guiding conditions to guide the flow matching model in generating candidate trajectories. The candidate trajectories undergo potential spatial perturbation stability verification, and a flow field velocity consistency score is calculated using a deterministic vector field to select the optimal trajectory in an unsupervised manner. The optimal trajectory is executed, and a post-event semantic verification and dynamic backtracking mechanism based on thought chains is introduced to achieve self-correction in task execution. This invention significantly improves the agent's 3D perception capability in uncertain environments, the diversity of task planning, the physical feasibility of actions, and the robustness of long-term task completion through the synergy of flow matching technology and autoregressive thought chains, achieving robust planning and execution of complex long-term tasks. Attached Figure Description
[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.
[0024] Figure 1 This is a flowchart illustrating the embodied intelligent agent training method based on flow matching to generate 3D perception and thought chain provided by the present invention.
[0025] Figure 2 This is a schematic diagram of the structure of the embodied intelligent agent training method based on flow matching to generate 3D perception and thought chain provided by the present invention.
[0026] Figure 3 The schematic diagram of the joint generation principle of three-dimensional latent representation and motion trajectory based on flow matching provided by the present invention.
[0027] Figure 4 The flowchart of the autoregressive thought chain dynamic programming provided by this invention. Detailed Implementation
[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and given in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.
[0029] The inventive concept of this invention is as follows: By directly learning 3D perceptual representations in the latent space through flow matching technology, the intelligent agent can efficiently model environmental geometry, object layout, and potential dynamic interaction relationships. Then, combined with a thought chain, the system can generate multiple heterogeneous inference paths, decompose complex long tasks using different semantic strategies, and generate corresponding continuous action trajectories for each strategy. During the inference phase, the system automatically filters physically unstable high-risk strategies based on a stability verification mechanism for latent space perturbations. In the action optimization stage, the deterministic vector field inherent in the flow matching model is used as the physical truth, and the optimal action with the smoothest physical execution and highest model confidence is determined by calculating the consistency of flow field velocity. Finally, the thought chain is used for post-event semantic verification and dynamic backtracking, achieving self-correction and robust implementation of complex long-term tasks.
[0030] like Figure 1 As shown in the embodiment, an embodied agent training method based on flow matching to generate 3D perception and thought chains includes the following steps: S1. Receive visual images and language commands, and generate a 3D geometric structure-aware representation in the latent space using a pre-trained flow matching model. This 3D geometric structure-aware representation is used to describe the spatial layout, depth relationships, and reachable areas of objects in the environment, as well as the robotic arm's reachability areas. During model training, this step introduces random masks and noise to ensure that the generated latent representation can accurately recover the spatial layout and depth relationships of objects in the environment even under partial occlusion or sensor noise interference.
[0031] In embodiment S1.1, the embodied intelligent agent receives visual images. With language instructions To achieve a unified representation of information from different modalities, features are first extracted using a pre-trained encoder and then mapped to a shared latent space through a projection layer. Specifically: like Figure 2 As shown, the visual encoder processes the visual image Processed into high-dimensional visual feature sequences , The total number of visual feature sequences. For visual feature sequences, the language model encoder will use language instructions. Mapped to semantic embedding sequence , The total number of semantic embedding sequences. For semantic embedding sequence dimension.
[0032] To eliminate dimensional differences between modalities, a multimodal projection layer is introduced to map the aforementioned features to a unified dimension. And then splice them together to form the original joint conditions. : , Should This ensures the consistency between visual perception and linguistic semantics at the underlying representation level, providing an aligned multimodal context for subsequent generation tasks.
[0033] S1.2 will combine the conditions The input is fed into a flow matching model, which learns a deterministic vector field to generate a real 3D geometric perception representation from the initial noise distribution. .
[0034] In the flow matching model construction and training phase, an anti-interference mechanism is introduced: a random mask generator is defined. Apply a mask or inject Gaussian noise to the input visual token portion. , Construct damaged joint conditions based on mask ratio. : .
[0035] By applying a random mask or injecting Gaussian noise to the input visual token, the model is forced to learn a vector field capable of recovering the complete 3D geometry from incomplete or noisy data.
[0036] Define the time step for stream matching Constructing from standard Gaussian noise distribution Linear interpolation path to real 3D annotation The training objective of the flow matching model is to optimize the network parameters. This makes the predicted vector field In the input of the damaged joint condition It can still restore the correct target flow direction. The loss function of the flow matching model is expressed as: , In the formula, The loss function for the flow matching model is... For mathematical expectation, It is the square of the L2 norm.
[0037] During the inference phase, even with interference in the input, a realistic 3D geometric structure perception representation is generated using the trained vector field and a numerical ordinary differential equation solver (ODE solver). .
[0038] S1.3 generates a real 3D geometric structure perception representation The environment is parsed into explicit context description information for use by downstream tasks. Define the environment decoder. This maps implicit features to a structured set of environmental elements. : , in, Indicates the spatial layout of objects. Indicates the distribution of depth relationships. This represents the reachability mask of the embodied intelligent agent's robotic arm. This set of environmental elements... With the original They are jointly transmitted to subsequent steps S2 and S3, providing a robust and geometrically well-defined perceptual basis for complex long-range task reasoning.
[0039] S2. Based on language instructions and 3D geometric structure perception representation, an autoregressive thought chain generator is used to generate multiple semantically heterogeneous reasoning paths. Each reasoning path is parsed into a structured semantic strategy, and the structured semantic strategy is fused with the 3D geometric structure perception representation to obtain multiple semantically heterogeneous conditions. These paths, while maintaining consistency with the final task objective, employ different semantic strategies to decompose long tasks. This thought chain sequence is not only used to plan the order of operations but also provides constraint logic information, such as physical limitations, object dependencies, and action sequence conditions. The final generated sub-task sequence is jointly encoded with 3D latent features to provide explicit semantic guidance for action candidates.
[0040] In embodiment S2.1, an autoregressive thought chain generator is used to construct diverse reasoning paths. Specifically, the pre-trained large language model is defined as follows: Receive language instructions and task context The task context is derived from the real 3D geometric structure perception representation. or collection of environmental elements In order to generate representations of different semantic strategies A set of heterogeneous reasoning sequences A divergent sampling strategy with a high temperature coefficient is adopted: .
[0041] The autoregressive generation process follows a chain rule, where each sequence It consists of a series of discrete subtask tokens. For the inference path sequence index, and and Significant semantic differences exist in the task decomposition logic, thus forming the initial heterogeneous branches of the mind tree. For example, for the instruction "clean up the desktop," the embodied agent may generate two different sub-task sequences: "efficiency-first strategy" and "security-first strategy," thereby providing rich initial branches for subsequent mind tree construction.
[0042] S2.2 For each reasoning path Further analysis and generation of structured key step descriptions Define a fine-grained description of the generation function. High-level strategy Mapped to tuples containing physical execution limits: , in Indicates the order of operations. Represents an object dependency graph. These descriptions represent spatial action constraints. For example, in a "safety-first" strategy, the description would explicitly include the sequential constraint of "removing obstacles before grabbing the target," while in an "efficiency-first" strategy, the obstacle avoidance step might be omitted. These descriptions ensure that each strategy is not only semantically coherent at the logical level but also implicitly contains specific physical execution constraints.
[0043] To use these symbolic descriptions for subsequent computation, a text encoder is used to transform them into a dense sequence of policy embedding vectors.
[0044] S2.3 will refer to the above Group Policy Embedded Vector Sequence Compare with the anti-interference three-dimensional geometric structure perception representation output in step S1. Joint encoding is performed to generate semantically heterogeneous conditions. A cross-modal attention mechanism is then used for 3D geometric structure-aware representation. For queries, embed vector sequences according to the strategy corresponding to the inference path. For keys and values, semantic constraints are injected into the geometric space: , , , Finally, the number was obtained The semantic heterogeneity conditions corresponding to this strategy: , Through this process, multiple sets of semantically heterogeneous conditions were constructed. These inputs share the same environmental geometry. Based on this, it carries a completely different semantic guidance signal, which serves as a guiding condition in step S3 to drive the flow matching model to generate motion trajectory candidates with different forms.
[0045] S3. Input the semantically heterogeneous conditions into the flow matching model to guide the generation of continuous action trajectories that match the structured semantic strategy, and then obtain candidate trajectories through kinematic screening. In this process, the flow matching vector field plans motion paths that conform to their respective physical constraints according to different semantic logics. Thanks to the generative model's ability to model uncertainty, the generated trajectories automatically smooth out minor control jitters while satisfying specific strategy semantics, ensuring the physical executability of the actions.
[0046] S3.1 as Figure 3 As shown, a conditional action vector field based on a specific policy is constructed in the three-dimensional latent space. The output of step S2 is received. Group semantic heterogeneity conditions For each strategy The vector field function learned by the flow matching model is This vector field defines the action state. With virtual generation time The evolution rate, i.e., the deterministic velocity field: , This equation describes the distribution from standard Gaussian noise. Distribute to target actions that conform to the specific policy semantics The direction of evolution. Due to Differences, vector fields The topology changes with the strategy, thereby ensuring the generation of action trends. Strictly follow the specific logical path of the thought chain plan.
[0047] S3.2 Numerical integration and trajectory smoothing are performed using an Ordinary Differential Equation Solver (ODE solver). This is achieved from standard Gaussian noise. Sample the initial state and use the ODE solver in the time domain Numerical integration is performed on each conditional vector field to solve for the continuous motion trajectory. : , During this integration process, the low-pass filtering characteristics of the ODE solver automatically smooth out potential high-frequency control noise, effectively suppressing minor jitter during robotic arm execution.
[0048] S3.3 Aggregates the initial candidate set and performs kinematic screening. Multiple motion trajectories generated based on different strategies are aggregated into an initial set. Define the basic kinematic constraint functions of the robotic arm. This function includes joint angle limits. Workspace Boundaries Singularity detection: , in, For time steps The trajectory of movement, This is the minimum limit for the joint angle. This sets the maximum limit for the joint angle.
[0049] This function is used to eliminate "degenerate solutions" that clearly violate physical constraints or are unreachable, thus obtaining a set of candidate trajectories. : , The candidate trajectory set As the input basis for perturbation consistency verification in the subsequent S4 step, it ensures that computational resources are used only to evaluate physically feasible trajectories.
[0050] S4. For each candidate trajectory, apply perturbation to the corresponding semantic heterogeneous conditions to guide the flow matching model to generate a verification trajectory. Based on the action trajectory and the verification trajectory, perform stability verification and screening to obtain a set of safe trajectories.
[0051] In embodiment S4.1, for each candidate trajectory and its corresponding semantic heterogeneity conditions A perturbation verification sample set was constructed to simulate sensor errors. With the uncertainty of semantic understanding ,generate A set of small random perturbation vectors (usually following a Gaussian distribution) ) or random mask matrix Inject these perturbations into the original semantic heterogeneous conditions. Generate a set of stress test conditions. : ,or .
[0052] S4.2 utilizes a flow matching model to perform parallel verification generation and self-consistency quantization. The above... A parallel input stream matching model for each stress test condition is used, and the corresponding set of verification trajectories is generated using the ODE solver. : .
[0053] Next, the original candidate trajectories are calculated. With each verification trajectory The degree of dispersion in geometric space. Define the perturbation consistency index. Mean squared error (MSE) or variance between trajectories: , This represents the square of the L2 norm.
[0054] This metric quantifies the self-consistency of a specific strategy under input perturbations: The smaller the value, the more likely the action strategy is located in the manifold stable region and has intrinsic robustness to uncertainty; The larger the value, the more likely the action is located at the edge of the manifold or in a dynamically unstable region, making it extremely sensitive to initial values and environmental noise.
[0055] S4.3 Executes a dynamic pruning strategy based on consistency metrics. A safety threshold is set. In practical applications, the security threshold A dynamic adaptive setting mechanism based on task granularity and environmental geometric features is adopted. Specifically, it is based on the set of environmental elements generated in step S1. The structured semantic policy description generated in step S2 is used to dynamically lower the safety threshold when the structured semantic policy indicates that the task is a high-precision operation, or when the three-dimensional geometric structure perception indicates that the current embodied agent's trajectory is close to the workspace boundary or obstacle; conversely, it is appropriately relaxed in open spaces or coarse-grained tasks. In a preferred embodiment, for action trajectories normalized to the range of [-1, 1], the baseline safety threshold is set to 0.01-0.05.
[0056] The candidate trajectory set is evaluated according to the following criteria. Filtering is performed to generate a clean set of safe tracks. : If the consistency variance of a candidate trajectory Trajectories below the safety threshold are deemed to possess intrinsic safety and are marked as "high-confidence candidates" for retention; conversely, trajectories with excessively large variance are automatically eliminated as potentially high-risk paths.
[0057] S5. Using the deterministic vector field learned by the flow matching model as a benchmark, calculate the consistency score between the actual motion direction of each trajectory in the safe trajectory set at each state point and the predicted direction of the deterministic vector field. Select the optimal action sequence based on the consistency score. By using the deterministic vector field learned by the flow matching model as the "physical truth," calculate the consistency of the flow field velocity between the actual tangent direction of each candidate trajectory at each time step and the flow direction predicted by the model. Based on the manifold assumption, the system determines that the trajectory with the smoothest and most confident motion at the physical level is the optimal solution under the current semantic conditions, and directly selects the trajectory with the highest consistency score as the optimal action sequence. Then, a final continuous action plan is formed to ensure the safe and efficient execution of multi-step tasks in complex environments.
[0058] In this embodiment, an action quality benchmark based on the training distribution is defined. Stream matching model The training phase aims to fit the distribution of data taught by experts. The target flow field is constructed. Its optimization objective is to minimize the conditional regression loss: , in, For conditional regression loss, For flow matching model, For the parameters of the flow matching model, For time steps, For mathematical expectation, For expert data distribution, For the target condition state, For time steps The interpolation state, Let this be the target vector field, used to predict the state from the current state. Correctly evolve to the target The ideal direction It is the square of the L2 norm.
[0059] After training convergence, the predicted field The probability distribution of actions conforming to physical laws and task specifications is explicitly encoded. If a trajectory is generated... The fact that the evolutionary direction is highly consistent with the data indicates that the action strictly falls within the high probability density region of the expert teaching data, possessing high quality and reliability.
[0060] S5.2 performs flow field velocity consistency calculations. For example... Figure 4 As shown, for the candidate set of safe trajectories filtered in step S4 For each trajectory (Discretize into time series) Perform a consistency metric. First, calculate the trajectory in... The actual tangent direction at that moment: ; At the same time, obtain the flow matching model at the current state point. and normalized time Predicted vector field flow direction: ; Next, the cosine similarity between the actual tangent direction and the predicted vector field direction is calculated, and the results are aggregated across the entire time step to obtain the flow field velocity consistency index. : , This indicator The degree of fit between the generated actions and the model's internalized expert teaching distribution was precisely quantified. It is an L1 norm. This is the smoothing constant.
[0061] S5.3 executes an unsupervised optimal strategy locking without requiring an external semantic model. All candidate action sequences are directly sorted based on the flow field velocity consistency score calculated above, and the safest trajectory with the highest score is defined. The optimal action sequence for the current subtask.
[0062] S6. Execute the optimal action sequence and use the autoregressive thought chain generator to perform semantic verification on the execution result. If the verification fails, select the suboptimal trajectory from the safe trajectory set and re-execute it to realize the training and dynamic backtracking of the embodied agent.
[0063] S6.1 Perform physical control and real-world feedback acquisition. Lock the optimal action sequence from step S5. The data is sent to the underlying controller of the embodied intelligent robotic arm to execute physical operations.
[0064] After the action is completed, the vision sensor is invoked to capture a real image of the current environment. This is the real image. It captures the actual physical changes that actions produce on the environment, serving as validation for the current thought tree node. Whether it is an effective objective feedback basis.
[0065] S6.2 Perform post-hoc semantic verification based on thought chains. Reuse the autoregressive thought chain generator based on the pre-trained large language model from step S2. Switch it to discrimination mode. Build a verification prompt. It includes a real-world image of the action after it has ended and the text target of the current subtask. : , The classification layer outputs the conditional probability of the current embodied agent's training task completion or directly generates a binary judgment result. : ,in This is the confidence threshold. If This indicates that the change in environmental conditions is in line with expectations; if This indicates that the physical execution of the action has been completed, but the semantic goal has not been achieved.
[0066] S6.3 performs dynamic programming and backtracking based on a ToT (Total Thinking Tree) structure. Based on the validation results... Update the global planning status. When the task is completed ( Confirm the current mind tree node If successful, mark it as complete. Proceed to the next subtask node and continue looping steps S2-S6. If the task fails, the current strategy branch is deemed invalid, automatic pruning is performed, and the candidate set is updated. The flow field velocity consistency score is retrieved from the candidate set. The second-highest action becomes the new optimal solution. Then roll back the state to before the action was executed and issue the command. Then, it re-enters the physical execution loop.
[0067] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A method for training embodied intelligent agents based on flow matching to generate 3D perception and thought chains, characterized in that, Includes the following steps: Step 1: Receive visual images and language instructions, and generate a three-dimensional geometric structure perception representation in the latent space through a pre-trained flow matching model; Step 2: Based on language instructions and three-dimensional geometric structure perception representation, use an autoregressive thought chain generator to generate multiple semantically heterogeneous reasoning paths, and parse each reasoning path into a structured semantic strategy. Then, fuse the structured semantic strategy with the three-dimensional geometric structure perception representation to obtain multiple semantically heterogeneous conditions. Step 3: Input the semantic heterogeneous conditions into the flow matching model to guide the generation of continuous action trajectories that match the structured semantic strategy, and then obtain candidate trajectories through kinematic screening; Step 4: For each candidate trajectory, apply perturbation to the corresponding semantic heterogeneous conditions to guide the flow matching model to generate a verification trajectory. Based on the action trajectory and the verification trajectory, perform stability verification and screening to obtain a set of safe trajectories. Step 5: Using the deterministic vector field learned by the flow matching model as a benchmark, calculate the consistency score between the actual motion direction of each trajectory in the safe trajectory set at each state point and the predicted direction of the deterministic vector field, and select the optimal action sequence based on the consistency score. Step 6: Execute the optimal action sequence and use the autoregressive thought chain generator to perform semantic verification on the execution result. If the verification fails, select the suboptimal trajectory from the safe trajectory set and re-execute it to realize the training and dynamic backtracking of the embodied agent.
2. The embodied agent training method based on flow matching to generate 3D perception and thought chain according to claim 1, characterized in that, The training process of the flow matching model in step 1 includes: A training sample set consisting of visual images, language instructions, and corresponding real 3D annotations is obtained. The visual images and language instructions in each training sample are extracted into visual features and language features respectively by the encoder, and then concatenated to form the original joint conditions. Construct a linear interpolation path from standard Gaussian noise distribution to real 3D annotation; In each training iteration, the original joint conditions are masked or injected with noise in conjunction with an anti-interference mechanism to establish impaired joint conditions. Using the instantaneous velocity of the linear interpolation path as the optimization objective, a flow matching model loss function is established to optimize the parameters of the flow matching model, resulting in the optimized flow matching model; wherein, the flow matching model loss function is expressed as: ， In the formula, The loss function for the flow matching model is... For the parameters of the flow matching model, For time steps, Standard Gaussian noise, For the perceived representation of the target's three-dimensional geometric structure. For the damaged joint conditions, From arrive The linear interpolation path, For flow matching model, For mathematical expectation, It is the square of the L2 norm.
3. The embodied agent training method based on flow matching to generate 3D perception and thought chain according to claim 1, characterized in that, Step 2, which involves generating multiple semantically heterogeneous reasoning paths using an autoregressive thought chain generator, includes: Using a pre-trained large language model as an autoregressive thought chain generator, with language instructions and three-dimensional geometric structure perception representation as conditional inputs, a high-temperature sampling strategy is adopted to perform multiple independent autoregressive generation, resulting in a set of multiple semantically heterogeneous reasoning path sequences, where each reasoning path sequence consists of a series of discrete subtask tokens.
4. The embodied agent training method based on flow matching to generate 3D perception and thought chain according to claim 3, characterized in that, The strategy of parsing each reasoning path into a structured semantic path includes: For each inference path sequence, a generator function is described in fine granularity. Parsing into a structured semantic strategy description: ,in Indicates the order of operations. Represents an object dependency graph. Represents spatial action constraints. Index for the reasoning path sequence; A text encoder is used to encode structured semantic policy descriptions into a sequence of policy embedding vectors.
5. The embodied agent training method based on flow matching to generate 3D perception and thought chain according to claim 4, characterized in that, In step 2, the fusion of multiple semantic heterogeneous conditions includes: using the three-dimensional geometric structure perception representation as the query, using the policy embedding vector sequence corresponding to the reasoning path as the key and value, and calculating the semantically injected three-dimensional geometric structure perception representation through a cross-modal attention mechanism; The semantically injected 3D geometric structure perception representation is fused with the original 3D geometric structure perception representation to form a semantically heterogeneous condition.
6. The embodied agent training method based on flow matching to generate 3D perception and thought chain according to claim 1, characterized in that, Step 3 specifically includes: For each semantically heterogeneous condition, the flow matching model defines a conditionally dependent deterministic velocity field; by sampling the initial action state from a standard Gaussian distribution and using a numerical ordinary differential equation solver to numerically integrate the deterministic velocity field in the time domain, a continuous action trajectory matching the structured semantic strategy is generated. Define the kinematic constraint function for the embodied intelligent agent. The kinematic constraint function includes joint angle restrictions, workspace boundaries, and singularity detection. The action trajectory is filtered by the kinematic constraint function to obtain candidate trajectories. The kinematic constraint function is expressed as: ， In the formula, For the trajectory of the action The kinematic constraint functions, For time step The trajectory of movement, This is the minimum limit for the joint angle. This is the maximum limit for the joint angle. This represents the workspace boundary.
7. The embodied agent training method based on flow matching to generate 3D perception and thought chain according to claim 1, characterized in that, Step 4 verifies the trajectory generation process, including: For each action trajectory and its corresponding semantic heterogeneous conditions, multiple sets of random perturbation vectors or random mask matrices are generated as perturbations. These perturbations are then injected into the original semantic heterogeneous conditions to obtain a set of stress test conditions. The random perturbation vectors follow a Gaussian distribution. Each condition in the set of stress test conditions is input into the flow matching model in parallel; the flow matching model generates the corresponding verification trajectory through an ordinary differential equation solver based on the same initial noise state, which is used for subsequent stability verification.
8. The embodied agent training method based on flow matching to generate 3D perception and thought chain according to claim 7, characterized in that, Step 4 involves stability verification and filtering based on the action trajectory and verification trajectory to obtain a set of safe trajectories, including: The perturbation consistency index between candidate trajectories and all verified trajectories is calculated. Combining three-dimensional geometric structure perception representation and structured semantic strategy, a safety threshold is dynamically set. If the perturbation consistency index of a candidate trajectory is lower than the threshold, it is determined to have intrinsic safety and is retained; otherwise, it is discarded to obtain a set of safe trajectories. The disturbance consistency index is expressed as: , Indicates the number of candidate trajectory samples. Indicates the first Candidate trajectories, No. The verification trajectory corresponding to each candidate trajectory. This indicates the index of the verification trajectory sample. This represents the square of the L2 norm.
9. The embodied agent training method based on flow matching to generate 3D perception and thought chain according to claim 1, characterized in that, Step 5, the training process using the deterministic vector field learned by the flow matching model as a benchmark, includes: Based on the distribution of expert data, an interpolation path from standard Gaussian noise to the target optimal action sequence is constructed for each training sample, and the instantaneous velocity of the interpolation path at each state point is defined as the target vector field. The flow matching model is trained by optimizing its parameters by minimizing the following conditional regression loss function, which is used to simulate the optimal action evolution implicit in the expert data and conforms to physical laws and task specifications: ， in, For conditional regression loss, For flow matching model, For the parameters of the flow matching model, For time steps, For mathematical expectation, For expert data distribution, For the target condition state, For time step The interpolation state, Let this be the target vector field, used to predict the state from the current state. Correctly evolve to the target The ideal direction It is the square of the L2 norm.
10. The embodied agent training method based on flow matching to generate 3D perception and thought chain according to claim 1, characterized in that, In step 5, the optimal action sequence is selected based on the consistency score, including: Discretize the safety trajectories in the set of safety trajectories into a time series; At each time step of the time series, the actual tangent direction of the safety trajectory is calculated, and the predicted vector field direction of the flow matching model at the current state point is obtained. Calculate the cosine similarity between the actual tangent direction and the predicted vector field direction, and aggregate the cosine similarities across all time steps to obtain a consistency score: ， In the formula, For the first Consistency score of each safety trajectory The fit of the internalized expert teaching distribution in the flow matching model was quantified; The total number of time steps. For safe trajectory in time step The actual tangent direction, For the flow matching model at time step and the current state point The direction of the predicted vector field at that location. It is an L1 norm. It is a smoothing constant; The safe trajectories in the set of safe trajectories are sorted according to their consistency scores, and the sequence with the highest consistency score is selected as the optimal action sequence.