Robot goal-condition trajectory generation and planning method based on flow matching

Abstract

The present application relates to the technical field of robots, and relates to a robot target condition trajectory generation and planning method based on flow matching. The state of a robot at a current time, a task target state and a cumulative return maximum value are obtained, and a condition vector is constructed. A plurality of noise trajectories are sampled from a standard Gaussian distribution, and the state of the initial time of each noise trajectory is replaced with the state of the robot at the current time. Each candidate trajectory is determined through a trained conditional velocity field network. For each candidate trajectory, a first step action of the candidate trajectory is determined, and a value score of the first step action is obtained based on a trained target condition value network. According to the order of the value scores of the first step actions, a target action to be executed by the robot is determined, the target action is sent to the robot, and after updating the state of the robot, the above steps are cyclically executed. The present scheme can improve trajectory reasoning speed and meet the real-time planning requirements of the robot.

Description

Technical Field

[0001] This specification relates to the field of robotics technology, and in particular to a method for generating and planning robot target condition trajectories based on flow matching. Background Technology

[0002] In the field of autonomous robot operation, motion planning is a core technology for achieving tasks such as grasping, handling, and navigation. For target-conditional tasks, the robot needs to learn planning strategies from offline demonstration datasets based on the given target state (such as target position and orientation) and generate a motion trajectory that can reach the target.

[0003] In recent years, trajectory planning methods based on generative models have gained widespread attention, especially methods represented by diffusion models. These methods can generate diverse, high-quality trajectories by modeling the distribution of trajectory datasets. However, this approach suffers from slow inference speed, making it difficult to meet real-time planning requirements. Diffusion models rely on stochastic differential equations to map noise to data distribution, and the inference process typically requires about 100 iterations of denoising. Typical methods take about 0.153–0.213 seconds to generate a single trajectory, which is far longer than the single-step decision time required for real-time robot control.

[0004] Therefore, this specification provides a method for generating and planning robot target condition trajectories based on flow matching. Summary of the Invention

[0005] This specification provides a method for generating and planning robot target condition trajectories based on flow matching, in order to partially solve the aforementioned problems existing in the prior art.

[0006] The following technical solution is adopted in this specification: This manual provides a method for generating and planning robot target condition trajectories based on flow matching, including: S1. Obtain the robot's current state, the preset task objective state, and the maximum cumulative reward; S2. Construct a condition vector based on the maximum cumulative reward and the preset task objective state; S3. Sample multiple noise trajectories from a standard Gaussian distribution; S4. Replace the initial state of each noise trajectory with the current state of the robot; S5. Input each replaced noise trajectory, the preset initial time, and the constructed conditional vector into the trained conditional velocity field network to obtain each candidate trajectory; S6. For each candidate trajectory, determine the first action of the candidate trajectory, and input the first action, the current state of the robot and the preset task target state into the trained target conditional value network to obtain the value score of the first action; S7. Based on the ranking of the value scores of each first step action, determine the target action to be sent to the robot, send the target action to the robot, and after updating the state of the robot, repeat steps S1 to S7 until the loop termination condition is met.

[0007] Based on the aforementioned technical methods, this solution employs a flow matching framework to generate candidate trajectories by solving Ordinary Differential Equations (ODEs) and utilizing a conditional velocity field network. This eliminates the need for multi-step random iteration denoising required by diffusion models, thus improving trajectory inference speed and meeting the robot's real-time planning needs. The generated trajectories not only "achieve the task target state" but also select actions for the robot to execute based on value scoring, avoiding redundant and detours. Multiple different candidate trajectories are generated for each decision, covering various possible motion modes or situations, and the best one can be selected for execution, avoiding suboptimal or dangerous actions caused by single-modal prediction. It exhibits strong resistance to execution errors, environmental dynamic disturbances, and model inaccuracies, and will not fail due to a single incorrect prediction.

[0008] Furthermore, obtaining the maximum cumulative return in S1 specifically includes: Obtain a historical trajectory dataset of the robot performing operation tasks. Each historical trajectory dataset includes at least the task target state and the robot's state and actions at each moment. For each historical trajectory in the historical trajectory dataset where the task target state is "reached the target", calculate the cumulative reward from each moment in the historical trajectory to reaching the target; and for each historical trajectory in the historical trajectory dataset where the task target state is "not reached the target", determine the state of the robot at a specific moment in the historical trajectory as a virtual target, and calculate the cumulative reward from each moment before the specific moment in the historical trajectory to reaching the virtual target. Determine the maximum cumulative return from all cumulative returns.

[0009] Based on the above technical means, a large number of "failed" trajectories in the historical trajectory dataset are relabeled as successful experiences with intermediate sub-targets as conditions, which significantly expands the effective training samples for subsequent training of the value network, enabling the value network to be fully trained in a sparse reward environment, effectively solving the sparse reward problem and improving data utilization.

[0010] Furthermore, the conditional velocity field network is trained using the following method, wherein: From the historical trajectory dataset, select multiple trajectory segments of preset length from each historical trajectory where the task target status is reached; For each trajectory segment, a noise segment with the same shape as that trajectory segment is sampled from a standard Gaussian distribution; Determine the random sampling time, and based on the random sampling time, the trajectory segment, and the noise segment, determine the training sample and the true velocity vector of the training sample; The training samples, the random sampling time, and the preset conditional vector are input into the conditional velocity field network to be trained to obtain the predicted velocity vector output by the conditional velocity field network to be trained, wherein the conditional vector includes the cumulative reward and the task target state. Based on the difference between the predicted velocity vector and the true velocity vector, the loss is determined, and the conditional velocity field network to be trained is trained with the goal of minimizing the loss.

[0011] Furthermore, the target conditional value network is trained using the following method, wherein: From the historical trajectory dataset, multiple state-action target triples are determined; wherein each state-action target triple includes the robot's state at a certain moment, the action performed, and the task target state of the historical trajectory to which it belongs at that moment, and the calculated cumulative reward from that moment to reaching the target or virtual target is used as the label of the state-action target triple; For each state-action-target triplet, the state-action-target triplet is input into the target conditional value network to be trained, and the predicted cumulative reward output by the target conditional value network to be trained is obtained. Based on the difference between the predicted cumulative reward and the label of the target triplet for the state action, the loss is determined, and the target conditional value network to be trained is trained with the goal of minimizing the loss.

[0012] Furthermore, before inputting each state-action-target triple into the target conditional value network to be trained, the method further includes: Based on the multiple state-action target triplets, multiple triplet pairs are determined. Each triplet pair includes a state-action target triplet labeled as the cumulative reward from the current moment to the arrival of the target and a state-action target triplet labeled as the cumulative reward from the current moment to the arrival of the virtual target. For each state-action-target triple, the state-action-target triple is input into the target conditional value network to be trained, and the predicted cumulative reward output by the target conditional value network is obtained, specifically including: For each triplet pair, each state-action-target triplet in the triplet pair is input into the target conditional value network to be trained, and the predicted cumulative reward output by the target conditional value network is obtained. Based on the difference between the predicted cumulative reward and the label of the state-action target triple, the loss is determined, and the target conditional value network to be trained is trained with the goal of minimizing the loss. Specifically, this includes: For the difference between the label of each state-action-target triplet in the triplet pair and the predicted cumulative reward corresponding to that state-action-target triplet, determine the loss; Based on each loss, a comprehensive loss is determined, and the objective conditional value network to be trained is trained with the goal of minimizing the comprehensive loss.

[0013] Furthermore, in S6, for each candidate trajectory, the first step of determining that candidate trajectory is specifically included: For each candidate trajectory, the first action of the candidate trajectory is determined by the trained Actor network.

[0014] Furthermore, the Actor network is trained using the following method, wherein: From the historical trajectory dataset, multiple state target triples are determined. Each state target triple includes the robot's state at a certain moment, the task target state of the historical trajectory to which it belongs at that moment, the robot's state at the next moment, and the robot's execution action at that moment is used as a label. For each state-target triplet, the state-target triplet is input into the Actor network to be trained to obtain the predicted execution action output by the Actor network to be trained. Based on the difference between the predicted action and the label of the target triplet, the loss is determined, and the Actor network to be trained is trained with the goal of minimizing the loss.

[0015] Furthermore, in S7, based on the ranking of the value scores of each first-step action, the target action sent to the robot for execution is determined, specifically including: The first step actions are sorted from highest to lowest value score, and the first step action with the highest value score is selected as the target action to be sent to the robot for execution.

[0016] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for generating and planning robot target condition trajectories based on flow matching.

[0017] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a method for generating and planning robot target condition trajectories based on flow matching.

[0018] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects: This scheme employs a flow matching framework to generate candidate trajectories by solving Ordinary Differential Equations (ODEs) and utilizing a conditional velocity field network. Unlike diffusion models, it eliminates the need for multi-step random iterations and denoising, thus improving trajectory inference speed and meeting the real-time planning requirements of robots. The generated trajectories not only achieve the task objective state but also select actions for the robot to perform based on value scoring, avoiding redundant or circuitous actions. Multiple different candidate trajectories are generated for each decision, covering various possible motion modes or situations, and the best one can be selected for execution, avoiding suboptimal or dangerous actions caused by single-modal prediction. It exhibits strong resistance to execution errors, dynamic environmental disturbances, and model inaccuracies, and will not fail due to a single incorrect prediction. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings: Figure 1 A flowchart illustrating the robot target condition trajectory generation and planning method based on flow matching provided in the embodiments of this specification; Figure 2 This manual provides a comparison chart of robot task success rates; Figure 3 This is an ablation experiment diagram illustrating the effect of horizontal line length h on TFM performance, provided in this specification. Figure 4 This is a schematic diagram of the robot target condition trajectory generation and planning device based on flow matching provided in this specification; Figure 5 This specification provides a corresponding Figure 1 A schematic diagram of the structure of an electronic device. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in this specification without creative effort are within the scope of protection of this application.

[0021] In embodiments of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0022] In the field of autonomous robot operation, motion planning is a core technology for achieving tasks such as grasping, handling, and navigation. For goal-conditional tasks, robots need to learn planning strategies from offline demonstration datasets based on a given goal state and generate motion trajectories that can reach the goal. Such tasks are usually modeled as offline goal-conditional reinforcement learning problems, which face core challenges such as sparse reward signals, multimodal data distribution, and real-time inference.

[0023] In recent years, trajectory planning methods based on generative models, especially those represented by diffusion models, have generated diverse, high-quality trajectories by modeling the distribution of trajectory datasets. Typical works include diffusion models and decision diffusion models. However, this approach has significant drawbacks: First, its inference speed is slow, making it difficult to meet real-time planning requirements. Second, offline sparse rewards lead to low data utilization. In offline goal-conditional reinforcement learning (offline GCRL) tasks, only trajectories that successfully reach the goal provide positive rewards, and a large amount of data in the dataset that "does not reach the goal" cannot be directly used for training, resulting in serious data waste. Third, its multi-task generalization ability is insufficient. When faced with multi-task scenarios where the target position changes with the task, the generalization performance of existing methods in large state spaces decreases significantly, making it difficult to flexibly adapt to diverse operational goals.

[0024] Therefore, this specification provides a method for generating and planning robot target condition trajectories based on flow matching.

[0025] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0026] Figure 1 The flowchart illustrating the robot target condition trajectory generation and planning method based on flow matching provided in the embodiments of this specification includes the following steps: S1: Obtain the robot's current state, the preset task objective state, and the maximum cumulative reward.

[0027] This specification describes the process of generating and planning robot target condition trajectories based on flow matching. In the embodiments described herein, this process can be executed by a server. However, this specification does not limit the type of device or platform from which this process is performed; for example, a personal computer, mobile terminal, or other such device or platform can also be used. For ease of description, the following description uses a server as the execution entity.

[0028] In one or more embodiments of this specification, the server can obtain the robot's current state, the preset task target state, and the maximum cumulative reward value. The robot's state includes the robot's joint angles, end effector pose, and environmental perception information.

[0029] The server can acquire historical trajectory datasets of the robot's operational tasks. Each historical trajectory dataset includes at least the task target state and the robot's state and actions at each moment. It can be defined as:

[0030] In the formula, where for The robot's state at all times (including joint angles, end effector pose, and environmental perception information). for The robot performs its actions at specific times. The mission objective status includes two categories: objective achieved and objective not achieved.

[0031] It can represent the trajectory length and historical trajectory. have That moment.

[0032] Therefore, for each historical trajectory in the historical trajectory dataset where the task objective status is "reaching the objective", the server can calculate the cumulative reward from each moment in that historical trajectory until reaching the objective.

[0033] The formula for calculating the cumulative reward upon reaching the goal is:

[0034] In the formula, For historical moments The cumulative return at time t represents the cumulative return from time t. At that time The cumulative reward (at the moment of reaching the goal). The preset discount factor, . For a pre-defined sparse reward function (such as one that can be set to reach the target) A positive value (e.g., +1) is given when the condition is met, otherwise 0 is given. Representing historical trajectory Sparse rewards of time.

[0035] Similarly, for each historical trajectory in the historical trajectory dataset where the task objective state is "not reached," the robot's state at a specific moment in that historical trajectory is determined as the virtual objective. The cumulative reward from each moment before that specific moment in the historical trajectory until reaching the virtual objective is calculated. Here, the specific moment can be an intermediate moment uniformly sampled from the time series of that historical trajectory, and the robot's state at this intermediate moment is used as the virtual objective. This historical trajectory was marked as a "virtual success" trajectory. .

[0036] The formula for calculating the cumulative reward for reaching the virtual goal is:

[0037] In the formula, For historical moments The cumulative return at time t represents the cumulative return from time t. At that time The cumulative reward (at the moment the virtual goal is reached). Calculated as above. The expression, The preset discount factor, . For a pre-defined sparse reward function (such as setting a virtual goal), A positive value (e.g., +1) is given when the condition is met, otherwise 0 is given. Representing historical trajectory Sparse rewards of time.

[0038] Therefore, the server can determine the maximum cumulative reward from the various cumulative rewards calculated at the point of calculation.

[0039] In this specification, a conditional velocity field network can be trained based on historical trajectory datasets to predict robot actions and generate robot trajectories.

[0040] The server can train the conditional velocity field network using the following method: From the historical trajectory dataset, select multiple trajectory segments of preset lengths from all historical trajectories where the task objective status is "reached the target". For each trajectory segment... Sample the trajectory segment from the standard Gaussian distribution. Noise fragments of the same shape , ~N(0,I). Then, according to t∈[0,1], the random sampling time t is determined, and based on the random sampling time t and the trajectory segment... and noise fragments Determine training samples and training samples The true velocity vector.

[0041] Among them, training samples The true velocity vector is Determine training samples The calculation expression is:

[0042] Therefore, the training samples By inputting a random sampling time t and a preset conditional vector c into the conditional velocity field network to be trained, the predicted velocity vector output by the conditional velocity field network can be obtained. The condition vector c includes the cumulative reward and the task objective state, c = (R exp, g), R exp The expected cumulative return.

[0043] Finally, based on the difference between the predicted velocity vector and the true velocity vector, the loss is determined, and the conditional velocity field network to be trained is trained with the goal of minimizing the loss.

[0044] One way to determine the loss is to minimize the Conditional Flow Matching Loss (CFM Loss):

[0045] It is worth noting that during the training of the conditional velocity field network to be trained, the conditional vector can be randomly discarded with probability p, so that the trained conditional velocity field network model has both unconditional and conditional generation capabilities.

[0046] Furthermore, in this specification, a target conditional value network can be trained based on historical trajectory datasets to predict the value of robot actions.

[0047] The server can train the objective conditional value network using the following methods: From the historical trajectory dataset, multiple state-action-target triples are identified, which are multiple... Each state-action target triplet includes the robot's state at a given moment. Execution of actions The task target state g of the historical trajectory to which the target or virtual target belongs at that moment is calculated, and the cumulative reward from that moment to reaching the target or virtual target is used as the label of the state-action-target triplet.

[0048] Next, for each state-action-target triplet, the triplet is input into the target conditional value network to be trained, and the predicted cumulative reward output by the target conditional value network is obtained. Then, based on the difference between the predicted cumulative reward and the label of the state-action-target triplet, the loss is determined, and the target conditional value network is trained with the minimum loss as the optimization objective.

[0049] Furthermore, we jointly optimize the training objective condition value network.

[0050] Specifically, based on multiple state-action-target triples, multiple triple pairs can also be determined. Each triple pair includes a state-action-target triple labeled with the cumulative reward from that moment to reaching the target. A state-action-target triplet labeled with the cumulative reward from that moment until the virtual target is reached. .

[0051] Therefore, for each triplet pair, each state-action-target triplet in the triplet pair can be input into the target conditional value network to be trained, and the predicted cumulative reward output by the target conditional value network can be obtained.

[0052] That is, for each state-action-target triplet in the triplet pair, the state-action-target triplet is input into the target conditional value network to be trained, and the target conditional value network outputs the predicted cumulative reward corresponding to the state-action-target triplet. Then, the loss is determined based on the difference between the label of each state-action-target triplet in the triplet pair and the predicted cumulative reward corresponding to the state-action-target triplet.

[0053] Since the triplet pair includes two state-action-target triplets, and two losses can be determined in the previous step, the comprehensive loss can be determined based on each loss. Then, the target conditional value network to be trained is trained with the comprehensive loss as the optimization objective.

[0054] Among them, the state-action-target triplet labeled with the cumulative reward from that moment to reaching the target. ,loss The calculation expression can be:

[0055] For the state-action-target triplet labeled with the cumulative reward from that moment to reaching the virtual goal. ,loss The calculation expression can be:

[0056] Therefore, considering the overall losses Possible forms:

[0057] In the formula, These are the preset hyperparameters. The trained target conditional value network can then be used to evaluate the value of state-action-target triples.

[0058] S2: Construct a condition vector based on the maximum cumulative reward and the preset task target state.

[0059] S3: Sample multiple noise trajectories from a standard Gaussian distribution.

[0060] S4: Replace the initial state of each noise trajectory with the current state of the robot.

[0061] In one or more embodiments of this specification, the server may construct a condition vector based on a determined maximum cumulative reward and a preset task objective state.

[0062] The server can then sample multiple noise trajectories from a standard Gaussian distribution, such as sampling N noise trajectories from a standard Gaussian distribution. N(0,I), i=1,...,N. It's worth noting that the shape of each noise trajectory is the same as the trajectory segment during training the conditional velocity field network. Then, the initial state of each noise trajectory is replaced with the robot's current state; that is, the state of the noise trajectory at its first moment (the first moment) is forcibly replaced with the robot's current observed state. .

[0063] S5: Input each replaced noise trajectory, the preset initial time, and the constructed conditional vector into the trained conditional velocity field network to obtain each candidate trajectory.

[0064] In one or more embodiments of this specification, the server can input each replaced noise trajectory, along with a preset initial time and a constructed conditional vector, into the trained conditional velocity field network to obtain each candidate trajectory.

[0065] Specifically, for each noise trajectory after replacement, numerical integration is performed using the trained conditional velocity field network, which can be achieved using the Euler method:

[0066] In the formula, starting from t=0 (i.e., the preset initial time), the replaced noise trajectory can be... Iterate step by step to (Step length) If the value is 0.05, a complete candidate trajectory is obtained. Candidate trajectories The format is the same as each historical trajectory in the historical trajectory dataset.

[0067] S6: For each candidate trajectory, determine the first action of the candidate trajectory, and input the first action, the current state of the robot and the preset task target state into the trained target conditional value network to obtain the value score of the first action.

[0068] In one or more embodiments of this specification, the server can determine the first action of each candidate trajectory, i.e., the robot's action at the first moment. This first action, the robot's current state obtained in S1, and the preset task target state are then combined and input into the trained target conditional value network to obtain the target conditional value network's estimate of "how much cumulative reward can be obtained if this action is performed now and the robot continues to follow this trajectory." This cumulative reward is used as the value score for the first action.

[0069] S7: Based on the ranking of the value scores of each first step action, determine the target action to be sent to the robot, send the target action to the robot, and after updating the state of the robot, repeat steps S1 to S7 until the loop termination condition is met.

[0070] In one or more embodiments of this specification, since multiple candidate trajectories are generated, multiple value scores for the first-step actions can be obtained through the target conditional value network. Therefore, the server can determine the first-step action to be sent to the robot for execution as the target action based on the ranking of the value scores of each first-step action, and send the target action to the robot. Furthermore, after updating the robot's state at the next moment, steps S1 to S7 are executed cyclically to calculate the robot's target action at the next moment until the loop termination condition is met.

[0071] For multiple first-step actions, the server can sort the first-step actions from highest to lowest value score and determine the first-step action with the highest value score as the target action to be sent to the robot for execution.

[0072] It is worth noting that the loop termination condition can be reaching the preset task target state or reaching the preset maximum number of planning steps or loop count.

[0073] In addition, the server can also train the Actor network using the following methods, as described in this specification: Identify multiple state-target triples from historical trajectory datasets. Each state-action-target quadruple includes the robot's state at a given moment. The task objective status of the historical trajectory to which this moment belongs. The robot's state at the next moment. And record the robot's actions at that moment. As a label; For each state-target triplet, the state-target triplet is input into the Actor network to be trained, and the predicted execution action output by the Actor network is obtained. Then, based on the difference between the predicted execution action and the label of the state-target triplet, the loss is determined, and the Actor network to be trained is trained with the minimum loss as the optimization objective.

[0074] Therefore, for each candidate trajectory, the server can determine the first step of the candidate trajectory through the trained Actor network.

[0075] For robots with many joint degrees of freedom (such as robotic arms with 7 or more degrees of freedom), Actor networks can be used to help estimate the actions to be performed.

[0076] In this specification, taking the example of a mobile robot performing a target-conditional navigation task in a two-dimensional maze environment, the robot's state at each moment may include the robot's two-dimensional position coordinates (x, y) and lateral and longitudinal velocities (v). x , v y (v) x Let v be the lateral velocity. yThe vertical velocity is represented by the action performed at each moment, which is a velocity increment (i.e., the increment of the lateral velocity and the increment of the vertical velocity). The target state is represented by two-dimensional target coordinates. A conditional velocity field network and a target conditional value network are trained using a mixed historical trajectory dataset containing both successful and unsuccessful target arrivals. For each decision step, the robot samples N=64 candidate trajectories. The target conditional value network selects the optimal trajectory, performs the first step, and then replans the trajectory until the target is reached or more than 1000 steps have been taken.

[0077] In this specification, a 7-DOF robotic arm performing a grasping and placing task is used as an example. The robot's state at each moment includes the end effector position / velocity, the angles / velocities of the 7 joints, and the position of the target object (total dimensions approximately 25). The actions executed at each moment are the end effector velocity and the gripper opening / closing command (dimension 4). The task target state is the object placement position (dimension 3). The training network includes a conditional velocity field network, a target conditional value network, and an Actor network.

[0078] In this specification, in a multi-task scenario, the task target state in the condition vector c is dynamically specified as the task changes, and the conditional velocity field network, the target conditional value network, and the Actor network can adapt to navigation tasks at different target locations without retraining.

[0079] based on Figure 1 The flow matching-based robot target condition trajectory generation and planning method presented here employs a flow matching framework to solve ordinary differential equations (ODEs) and utilizes a conditional velocity field network to generate candidate trajectories. Unlike diffusion models, this eliminates the need for multi-step random iteration denoising, thus improving trajectory inference speed and meeting the real-time planning requirements of robots. The generated trajectories not only "achieve the task target state" but also select robot actions based on value scoring, avoiding redundant and detours. Multiple different candidate trajectories are generated for each decision, covering various possible motion modes or situations, and the best one can be selected for execution, avoiding suboptimal or dangerous actions caused by single-modal prediction. It exhibits strong resistance to execution errors, environmental dynamic disturbances, and model inaccuracies, and will not fail due to a single incorrect prediction. Experiments show that the time for a single trajectory generation is only about 0.017–0.024 seconds, approximately 10 times faster than the diffusion model (0.153–0.213 seconds), supporting online real-time planning for robots at frequencies ≥20 Hz.

[0080] Furthermore, by relabeling a large number of "failed" trajectories in the historical trajectory dataset as successful experiences conditioned on intermediate sub-objectives, the effective training samples for subsequent training of the value network are significantly expanded. This allows the value network to be fully trained in a sparse reward environment, effectively solving the sparse reward problem and improving data utilization. The flow matching velocity field directly fits the true distribution of the dataset through linear interpolation paths, naturally possessing the ability to model multi-peak distributions. It can generate a set of candidate trajectories covering multiple feasible paths, providing rich exploration space for robot decision-making.

[0081] Meanwhile, it achieves excellent performance in 2D maze navigation (Maze2D), multi-task navigation (Multi2D), and 7-DOF robotic arm operations (FetchReach, FetchPush, FetchPickAndPlace), demonstrating strong generalization and transferability. It is versatile for both single and multi-task operations and adaptable to various robot operation scenarios. The modular framework facilitates integration into existing robot systems. The flow matching generation model (conditional velocity field network) is decoupled from the target conditional value network and Actor network, allowing for independent replacement or upgrades. This facilitates adaptation to different robot platforms (mobile chassis, robotic arms, humanoid robots, etc.) and task types.

[0082] Figure 2 This is a comparison chart of robot task success rates provided in this manual. Figure 2 As shown, the horizontal axis lists the various robot tasks, including 2D maze navigation (Maze2D), multi-task navigation (Multi2D), and 7-DOF robotic arm operations (reaching FetchReach, pushing FetchPush, grasping and placing FetchPickAndPlace). The vertical axis represents the success rate. Each robot task uses five methods for trajectory generation and planning. The bars, from left to right, gradually darken in color, corresponding to Diffuser, Decision Diffusion (DD), Hierarchical Diffusion Model for Imitation (HDMI), Skill-based Sequential Diffusion (SSD), and TFM, representing this solution. Figure 2 It can be seen that this solution has the highest success rate in all robot tasks.

[0083] Figure 3 This is an ablation experiment diagram illustrating the effect of horizontal line length h on TFM performance, provided in this specification. TFM is... Figure 1The diagram shows an abbreviation for a flow-matching-based robot target condition trajectory generation and planning method. The horizontal axis, h, represents the trajectory length, and the vertical axis represents the success rate. As h increases, the success rates of the three robot tasks (Maze2D, Multi2D, and FetchPickAndPlace) first increase and then tend to stabilize. The experiment selected h = 64 as the default value.

[0084] Based on the flow matching-based robot target condition trajectory generation and planning method provided in one or more embodiments of this specification, following the same approach, this specification also provides a corresponding flow matching-based robot target condition trajectory generation and planning device, such as... Figure 4 As shown.

[0085] Figure 4 The schematic diagram of the robot target condition trajectory generation and planning device based on flow matching provided in this specification specifically includes: The acquisition module 400 is used to acquire the robot's current state, the preset task target state, and the maximum cumulative reward value. Construction module 402 is used to construct a condition vector based on the maximum cumulative reward and the preset task target state; Sampling module 404 is used to sample multiple noise trajectories from a standard Gaussian distribution; Replacement module 406 is used to replace the initial state of each noise trajectory with the current state of the robot. The trajectory generation module 408 is used to input each replaced noise trajectory, the preset initial time, and the constructed conditional vector into the trained conditional velocity field network to obtain each candidate trajectory. The value prediction module 410 is used to determine the first step of each candidate trajectory, and input the first step, the current state of the robot and the preset task target state into the trained target conditional value network to obtain the value score of the first step. The action execution module 412 is used to determine the target action to be sent to the robot for execution based on the ranking of the value scores of each first step action, send the target action to the robot, and after updating the state of the robot, cyclically execute the above modules until the loop termination condition is reached.

[0086] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 A method for generating and planning robot target condition trajectories based on flow matching is provided.

[0087] This instruction manual also provides Figure 5The diagram shows a schematic structural representation of the electronic device. Figure 5 As shown, at the hardware level, this electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above. Figure 5 A method for generating and planning robot target condition trajectories based on flow matching is provided.

[0088] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0089] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0090] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0091] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0092] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0093] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0094] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0095] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0096] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0097] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0098] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0099] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic or disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0100] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0101] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0102] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0103] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0104] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A method for generating and planning robot target condition trajectories based on flow matching, characterized in that, include: S1. Obtain the robot's current state, the preset task objective state, and the maximum cumulative reward; S2. Construct a condition vector based on the maximum cumulative reward and the preset task objective state; S3. Sample multiple noise trajectories from a standard Gaussian distribution; S4. Replace the initial state of each noise trajectory with the current state of the robot; S5. Input each replaced noise trajectory, the preset initial time, and the constructed conditional vector into the trained conditional velocity field network to obtain each candidate trajectory; S6. For each candidate trajectory, determine the first action of the candidate trajectory, and input the first action, the current state of the robot and the preset task target state into the trained target conditional value network to obtain the value score of the first action; S7. Based on the ranking of the value scores of each first step action, determine the target action to be sent to the robot, send the target action to the robot, and after updating the state of the robot, repeat steps S1 to S7 until the loop termination condition is met.

2. The method for generating and planning robot target conditional trajectories based on flow matching as described in claim 1, characterized in that, S1 obtains the maximum cumulative return, specifically including: Obtain a historical trajectory dataset of the robot performing operation tasks. Each historical trajectory dataset includes at least the task target state and the robot's state and actions at each moment. For each historical trajectory in the historical trajectory dataset where the task target state is "reached the target", calculate the cumulative reward from each moment in the historical trajectory to reaching the target; and for each historical trajectory in the historical trajectory dataset where the task target state is "not reached the target", determine the state of the robot at a specific moment in the historical trajectory as a virtual target, and calculate the cumulative reward from each moment before the specific moment in the historical trajectory to reaching the virtual target. Determine the maximum cumulative return from all cumulative returns.

3. The method for generating and planning robot target conditional trajectories based on flow matching as described in claim 2, characterized in that, The conditional velocity field network is trained using the following method, wherein: From the historical trajectory dataset, select multiple trajectory segments of preset length from each historical trajectory where the task target status is reached; For each trajectory segment, a noise segment with the same shape as that trajectory segment is sampled from a standard Gaussian distribution; Determine the random sampling time, and based on the random sampling time, the trajectory segment, and the noise segment, determine the training sample and the true velocity vector of the training sample; The training samples, the random sampling time, and the preset conditional vector are input into the conditional velocity field network to be trained to obtain the predicted velocity vector output by the conditional velocity field network to be trained, wherein the conditional vector includes the cumulative reward and the task target state. Based on the difference between the predicted velocity vector and the true velocity vector, the loss is determined, and the conditional velocity field network to be trained is trained with the goal of minimizing the loss.

4. The method for generating and planning robot target conditional trajectories based on flow matching as described in claim 2, characterized in that, The target conditional value network is trained using the following method, wherein: From the historical trajectory dataset, multiple state-action target triples are determined; wherein each state-action target triple includes the robot's state at a certain moment, the action performed, and the task target state of the historical trajectory to which it belongs at that moment, and the calculated cumulative reward from that moment to reaching the target or virtual target is used as the label of the state-action target triple; For each state-action-target triplet, the state-action-target triplet is input into the target conditional value network to be trained, and the predicted cumulative reward output by the target conditional value network to be trained is obtained. Based on the difference between the predicted cumulative reward and the label of the target triplet for the state action, the loss is determined, and the target conditional value network to be trained is trained with the goal of minimizing the loss.

5. The method for generating and planning robot target conditional trajectories based on flow matching as described in claim 4, characterized in that, Before inputting each state-action-target triple into the target conditional value network to be trained, the method further includes: Based on the multiple state-action target triplets, multiple triplet pairs are determined. Each triplet pair includes a state-action target triplet labeled as the cumulative reward from the current moment to the arrival of the target and a state-action target triplet labeled as the cumulative reward from the current moment to the arrival of the virtual target. For each state-action-target triple, the state-action-target triple is input into the target conditional value network to be trained, and the predicted cumulative reward output by the target conditional value network is obtained, specifically including: For each triplet pair, each state-action-target triplet in the triplet pair is input into the target conditional value network to be trained, and the predicted cumulative reward output by the target conditional value network is obtained. Based on the difference between the predicted cumulative reward and the label of the state-action target triple, the loss is determined, and the target conditional value network to be trained is trained with the goal of minimizing the loss. Specifically, this includes: For the difference between the label of each state-action-target triplet in the triplet pair and the predicted cumulative reward corresponding to that state-action-target triplet, determine the loss; Based on each loss, a comprehensive loss is determined, and the objective conditional value network to be trained is trained with the goal of minimizing the comprehensive loss.

6. The method for generating and planning robot target condition trajectories based on flow matching as described in claim 2, characterized in that, In S6, for each candidate trajectory, the first step of determining that candidate trajectory is specifically included: For each candidate trajectory, the first action of the candidate trajectory is determined by the trained Actor network.

7. The method for generating and planning robot target conditional trajectories based on flow matching as described in claim 6, characterized in that, The Actor network is trained using the following method, wherein: From the historical trajectory dataset, multiple state target triples are determined. Each state target triple includes the robot's state at a certain moment, the task target state of the historical trajectory to which it belongs at that moment, the robot's state at the next moment, and the robot's execution action at that moment is used as a label. For each state-target triplet, the state-target triplet is input into the Actor network to be trained to obtain the predicted execution action output by the Actor network to be trained. Based on the difference between the predicted action and the label of the target triplet, the loss is determined, and the Actor network to be trained is trained with the goal of minimizing the loss.

8. The method for generating and planning robot target conditional trajectories based on flow matching as described in claim 1, characterized in that, In S7, the target actions sent to the robot for execution are determined based on the ranking of the value scores of each first-step action. These actions include: The first step actions are sorted from highest to lowest value score, and the first step action with the highest value score is selected as the target action to be sent to the robot for execution.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1 to 8.

10. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in any one of claims 1 to 8.

Images