A Robotic Arm Path Planning Method Based on Transformer and Diffusion Model

Abstract

This invention discloses a robotic arm path planning method based on Transformer and diffusion models, belonging to the field of robotics technology. The method includes: collecting environmental information to generate a reference path and constructing a training dataset; constructing a conditional diffusion Transformer prediction network, extracting features, and achieving multimodal path prediction; applying Gaussian noise to the reference path and denoising the conditional diffusion Transformer prediction network; combining the diffusion model and a cost-guided mechanism to optimize the noisy path sampled from the Gaussian distribution until a smooth, collision-free path is generated; evaluating candidate paths and generating robotic arm joint control commands; the robotic arm executing the planned trajectory and performing real-time perception and online replanning. This invention organically combines environmental perception, Transformer encoding, and diffusion generation, rapidly generating multiple feasible trajectories that are both smooth and obstacle-avoiding in complex obstacle scenarios. It has good generalization ability and can adapt to different scenarios and dimensional changes.

Description

Technical Field

[0001] This invention relates to the field of robotics, and in particular to a method for planning the path of a robotic arm based on the Transformer and diffusion model. Background Technology

[0002] Path planning is a key technology for achieving autonomous movement in robotic systems and has been widely applied in scenarios such as industrial manufacturing, warehousing and logistics, autonomous driving, and emergency rescue. Traditional graph search and optimization methods can generate feasible paths in static or regularized environments, but when faced with high-dimensional scenarios, they often require significant preprocessing or online replanning costs. Furthermore, because the path generation process involves cascading decisions, small errors can easily accumulate, leading to a decrease in the smoothness and feasibility of the final trajectory.

[0003] Furthermore, existing learning-based path planning algorithms typically rely on fixed scenario data during model training, resulting in limited generalization ability and insufficient rapid adaptability to new environments. This makes it difficult to meet the dual requirements of real-time performance and robustness in complex and ever-changing production or operational environments.

[0004] The recently emerging diffusion model offers a new approach to generative modeling. It simulates data distribution through progressive denoising, directly generating complete trajectory sequences in a high-dimensional continuous space, and naturally possesses smooth and coherent characteristics. Introducing such models into the field of path planning holds promise for overcoming the bottlenecks of traditional methods in long-term dependencies and error accumulation.

[0005] However, in the implementation process, key issues such as the effective expression of environmental constraints, efficient sampling of high-dimensional action spaces, and precise guidance of target conditions need to be considered. In particular, in practical deployment, how to maintain model generation efficiency while taking into account environmental adaptability and result controllability is a core challenge that current research urgently needs to address.

[0006] Therefore, there is an urgent need for a novel framework that can capture the spatiotemporal characteristics of the environment and efficiently generate globally optimal or near-optimal paths under target guidance, in order to improve the planning accuracy and execution reliability of robots in complex scenarios. This framework should not only support multidimensional representations of obstacle distribution and constraints, but also have the ability to conditionally modulate the generated results, thereby achieving efficient and robust path generation and online adjustment in uncertain environments. Summary of the Invention

[0007] The purpose of this invention is to provide a robotic arm path planning method based on Transformer and diffusion model, which can organically combine environmental perception, Transformer encoding and diffusion generation, and can quickly generate multiple feasible trajectories that are both smooth and obstacle-avoiding in complex obstacle scenarios. It has good generalization ability and can adapt to different scenarios and dimensional changes.

[0008] To achieve the above objectives, this invention provides a robotic arm path planning method based on the Transformer and diffusion model, comprising the following steps:

[0009] S1. Randomly sample different obstacle configurations to initialize the environment and record environmental information. Use a hybrid planning strategy to generate reference paths in two-dimensional and three-dimensional scenes: First, obtain a rough initial trajectory through RRT-Connect, and then refine and smooth it with the help of the GPMP2 optimizer to build a training dataset containing obstacle and start-end information.

[0010] S2. Construct a conditional diffusion Transformer prediction network, which consists of three parts: a path encoder, a spatiotemporal awareness environment encoder, and a conditional diffusion Transformer decoder. The path encoder and spatiotemporal awareness environment encoder are used to extract obstacle geometric features and trajectory global dependency features. The conditional diffusion Transformer decoder is used to fuse the above features to achieve multimodal path prediction.

[0011] S3. Apply Gaussian noise to the reference path and use a conditional diffusion Transformer prediction network for stepwise denoising training, so that the conditional diffusion Transformer prediction network learns the denoising distribution under each diffusion step.

[0012] S4. In each time step of the denoising training, combine the diffusion model and the cost-guided mechanism to optimize the noise path sampled from the Gaussian distribution;

[0013] S5. Repeat S4 until a smooth, collision-free path is obtained;

[0014] S6. The path with the shortest smoothness and path length is selected as the optimal trajectory. The generated multi-candidate paths are evaluated by multiple objectives and transformed by inverse kinematics to generate joint control commands for the robotic arm.

[0015] S7: The robotic arm executes the planned trajectory and ensures safety and accuracy in dynamic environments through real-time sensing and online replanning.

[0016] Preferably, in S2, the path encoder performs feature extraction by including the following steps:

[0017] S201. Represent the input path sequence as a shape of... B × N × D The three-dimensional tensor, in which B For sample batch size, N The number of path points. D The original feature dimensions for each path point;

[0018] S202. The original features of each path point are upscaled using the channel mapping module, and a multilayer perceptron structure is used to increase the dimensionality of the input from dimensionality. D Projected onto the middle channel dimension C The resulting shape is B × N The representation of ×C;

[0019] S203, Input to a stacked 3-layer Mixer module, along the path point dimension of each layer. N Establish sequential relationships between path points to capture global structural features within the path;

[0020] S204. The first channel-dimensional MLP module is used to fine-tune the features of each path point to enhance its local semantic expression capability.

[0021] S205. Insert a regularization layer before the first and second MLP modules to enhance network stability and generalization ability;

[0022] S206. Combine the information fusion results of the channel direction and the token direction by using the residual connection splicing method;

[0023] S207, Embed information into the second MLP module. H The final path encoding representation is obtained from the 3D latent space, and the output is: B × N × H The path feature tensor of the shape.

[0024] Preferably, in S2, the spatiotemporal awareness environment encoder performs feature extraction by including the following steps:

[0025] S211. Represent the input environmental information as a shape of... The three-dimensional tensor, in which Indicates batch size. This indicates the number of environmental targets in each sample. Represents the feature dimensions of each target;

[0026] S212. Map the time step corresponding to each sample to an MLP module. 3D time feature vector;

[0027] S213, Embed the original features of each environmental target into another MLP module. The latent space is then mapped through a linear layer to... Dimension, and then transform it through a mapping layer. The features of trajectory points are obtained, resulting in a dimension of... The characteristic tensor;

[0028] S214, will The feature tensors are input to the environment aggregation module, which includes sequentially connecting the dimensions from... Mapped to Linear layers, ReLU activation function, from Mapped to Linear layers, multi-head attention mechanisms, and The attention features are aggregated by a feature pooling layer, and the pooled global features are mapped back through a linear layer. Dimension A hidden layer set for the environment aggregation module;

[0029] S215. The environmental aggregation result is fused with the temporal embedding feature, and the fused encoding result is output as a shape. The spatiotemporal perception environment representation.

[0030] Preferably, in S2, the conditional diffusion Transformer decoder fuses obstacle geometric features and trajectory global dependency features, including the following steps:

[0031] S221. The path features output by the path encoder are embedded from bottom to top through position encoding to enhance the model’s perception of temporal and spatial location. The time-environment fusion encoding produced by the spatiotemporal perception environment encoder is extracted in parallel as conditional information.

[0032] S222. After the initial diffusion trajectory noise is fused with the conditional information in the affine modulation layer, it is input into the multi-layer stacked conditional diffusion Transformer module along with the position encoding features.

[0033] S223. In each layer of the Conditional Diffusion Transformer module, the Conditional Diffusion Transformer module normalizes the input features and injects conditional information through the affine modulation layer. It then enters the attention sub-layer and sequentially performs connection-gated attention, multi-head self-attention, and multi-head cross-attention. After each sub-module outputs, it first performs affine modulation and then adds the residual to the corresponding input.

[0034] S224. Complete attention processing, normalize, and introduce conditions through an affine modulation layer. Then, feed the data into a point-based feedforward network for high-dimensional feature mapping. The feedforward output also undergoes the same process and is added to the feedforward input residual. The attention and feedforward stages are stacked alternately in each layer. The decoder continuously fuses the trajectory and spatiotemporal conditions to iteratively predict the path of the next time series.

[0035] Preferably, in S3, a standard forward diffusion process is used on the dataset to progressively add noise to the original path. The noise addition formula is as follows:

[0036] ;

[0037] in, Represents the original path. Representative process Noise path at each time step This represents a pre-defined noise ratio that gradually increases in a cosine manner.

[0038] Preferably, in S3, the noise path is denoised during the training phase. diffusion time step The trajectory features generated by the path encoder and the scene embedded by the spatiotemporally aware environment encoder are input together into the denoising network. , by denoising network Predicting added noise The training objective is to minimize the mean square error between the predicted noise and the actual noise.

[0039] ;

[0040] The training process employs the AdamW optimizer, gradient pruning, and cosine annealing learning rate scheduling.

[0041] Preferably, in S3, the training process of the conditional diffusion Transformer prediction network includes the following steps:

[0042] S301. Prepare training data: Path datasets generated using the RRT-Connect algorithm and optimized using the GPMP2 optimizer, along with geometric information of obstacles in the corresponding scene, are organized into training samples in batches of 16, and the coordinates and dimensions are uniformly normalized.

[0043] S302. Constructing Noise Samples: In each training iteration, the number of diffusion steps is randomly selected, and the cumulative noise weight is calculated according to the predefined noise schedule. Noisy path points are generated through the forward diffusion process.

[0044] S303, Encoding condition input: The noisy path points and environmental information are fed into the path encoder and the spatiotemporal awareness environment encoder respectively to obtain path features and environmental features;

[0045] S304, Denoising Network Forward: Embed the path features, environmental features, and the time step of the diffusion steps into the input conditional diffusion Transformer decoder to predict the noise residual;

[0046] S305. Calculate the loss: Evaluate the difference between the model prediction and the actual noise based on the mean square error.

[0047] S306, Gradient Backpropagation: Using the AdamW optimizer, backpropagation is performed to update the model parameters under the constraint that the gradient norm does not exceed 1.0;

[0048] S307, Training Control: The model parameters are smoothed using exponential moving average, and the learning rate is adjusted according to the cosine annealing strategy.

[0049] S308. Iteration Termination: Repeat steps S302 to S307 until the predetermined number of training steps is completed, and save the training summary and model checkpoints at the preset frequency.

[0050] Preferably, in S4, the introduced cost-incentive mechanism includes collision costs. and smoothing costs These two cost functions are gradient-adjusted for the mean of the inverse diffusion prediction according to the following formula:

[0051] ;

[0052] ;

[0053] in, and These are coefficients representing collision cost and smoothing cost, respectively. The step size factor is proportional to the current noise variance.

[0054] Preferably, in S4, the collision cost, used to measure the proximity of the generated path to obstacles, is expressed as follows:

[0055] Collision costs This is used to measure the proximity of the generated path to obstacles, and its expression is as follows:

[0056] ;

[0057] in, This indicates the position of the path at N discrete time steps. Represents the Euclidean distance between the path point and the obstacle. represent A single path point in, where, ; The default safety distance is set; when the Euclidean distance between the path point and the obstacle is less than the safety distance, the collision cost is 0; when the distance is greater than the safety distance, the collision cost is the difference between the two.

[0058] Smoothing costs The expression used to constrain the second-order smoothness of the path is as follows:

[0059] ;

[0060] It is used to penalize abrupt changes between adjacent points and promote trajectory smoothing.

[0061] Therefore, this invention provides a robotic arm path planning method based on Transformer and diffusion model. It utilizes the RRT-Connect algorithm and GPMP2 optimizer to generate smooth and feasible reference paths in 2D and 3D scenes, and performs post-processing on the paths to construct a training dataset containing obstacle and origin / endpoint information. A path encoder and a spatiotemporal awareness environment encoder are constructed to extract obstacle geometric features and trajectory global dependency features. These features are then fused using a conditional diffusion Transformer decoder to achieve multimodal path prediction. Gaussian noise is applied to the reference paths, and a conditional diffusion Transformer prediction network is used for progressive denoising training, enabling the network to learn the denoising distribution at each diffusion step. At each time step of the inference process, the diffusion model and cost-guided mechanism are combined to optimize the noisy path sampled from the Gaussian distribution, repeating this process until a smooth, collision-free path is obtained. The final generated multi-candidate paths are then subjected to multi-objective evaluation and inverse kinematics transformation to select the optimal trajectory and generate robotic arm joint control commands. The robotic arm executes the planned trajectory, and real-time perception and online replanning ensure safety and accuracy in dynamic environments. This method can organically combine environmental perception, Transformer encoding, and diffusion generation, enabling it to quickly generate multiple feasible trajectories that are both smooth and obstacle-avoiding in complex obstacle scenarios. It has good generalization ability and can adapt to different scenarios and dimensional changes.

[0062] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0063] Figure 1 This is an overall flowchart of a robotic arm path planning method based on Transformer and diffusion model according to the present invention;

[0064] Figure 2 This is a schematic diagram of the dataset configuration of this invention;

[0065] Figure 3 This is a schematic diagram of feature extraction performed by the path encoder of the present invention;

[0066] Figure 4 This is a structural diagram of the spatiotemporal sensing environment encoder of the present invention;

[0067] Figure 5 This is a schematic diagram of the conditional diffusion Transformer decoder structure of the present invention;

[0068] Figure 6 This is a schematic diagram of the conditional guidance structure of the present invention;

[0069] Figure 7This is a schematic diagram of a robotic arm path planning method based on Transformer and diffusion model according to the present invention. Detailed Implementation

[0070] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0071] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0072] Example 1

[0073] This invention provides a robotic arm path planning method based on the Transformer and diffusion model, the overall flowchart of which is shown below. Figure 1 As shown, the specific process includes the following steps:

[0074] S1. Randomly sample different obstacle configurations to initialize the environment and record environmental information. A hybrid planning strategy is used to generate reference paths in both 2D and 3D scenes: First, a coarse initial trajectory is obtained through RRT-Connect, then refined and smoothed using the GPMP2 optimizer to construct a training dataset containing obstacle and start / endpoint information. To ensure the diversity and richness of the training data, this embodiment randomly generates 1000 independent scenes, sets 100 sets of start / endpoints as hard constraints in each scene, and plans 10 paths for each configuration, thus providing sufficient and reliable training samples for the diffusion model. The training samples are as follows: Figure 2 As shown.

[0075] S2. Construct a conditional diffusion Transformer prediction network, which consists of three parts: a path encoder, a spatiotemporal awareness environment encoder, and a conditional diffusion Transformer decoder. The path encoder and spatiotemporal awareness environment encoder are used to extract obstacle geometric features and trajectory global dependency features. The conditional diffusion Transformer decoder is used to fuse the above features to achieve multimodal path prediction.

[0076] Path encoder performs feature extraction, such as Figure 3 As shown, it includes the following steps:

[0077] S201. Represent the input path sequence as a shape of... B × N × D The three-dimensional tensor B For sample batch size, N The number of path points. D The original feature dimensions for each path point;

[0078] S202. The original features of each path point are upscaled using the channel mapping module, and a multilayer perceptron structure is used to increase the dimensionality of the input from dimensionality. D Projected onto the middle channel dimension C The resulting shape is B × N × C The representation of;

[0079] S203, Input to a stacked 3-layer Mixer module, along the path point dimension of each layer. N Establish sequential relationships between path points to capture global structural features within the path;

[0080] S204. The first channel-dimensional MLP module is used to fine-tune the features of each path point to enhance its local semantic expression capability.

[0081] S205. Insert a regularization layer before the first and second MLP modules to enhance network stability and generalization ability;

[0082] S206. Combine the information fusion results of the channel direction and the token direction by using the residual connection splicing method;

[0083] S207, Embed information into the second MLP H The final path encoding representation is obtained from the 3D latent space, and the output is: B × N × H The path feature tensor of the shape.

[0084] Spatiotemporal awareness environment encoders, such as Figure 4 As shown, feature extraction includes the following steps:

[0085] S211. Represent the input environmental information as a shape of... The three-dimensional tensor, in which Indicates batch size. This indicates the number of environmental targets in each sample. Represents the feature dimensions of each target;

[0086] S212. Map the time step corresponding to each sample to an MLP module. 3D time feature vector;

[0087] S213, Embed the original features of each environmental target into another MLP module. The latent space is then mapped through a linear layer to... Dimension, and then transform it through a mapping layer. The features of trajectory points are obtained, resulting in a dimension of... The characteristic tensor;

[0088] S214. This feature is input to the environment aggregation module, which includes sequentially connecting dimensions from... Mapped to Linear layers, ReLU activation function, from Mapped to Linear layers, multi-head attention mechanisms, and feature pooling layers are all important components of this system. The attention features are aggregated, and the pooled global features are mapped back through a linear layer. Dimension A hidden layer set for the environment aggregation module;

[0089] S215. The environmental aggregation result is fused with the temporal embedding feature, and the fused encoding result is output as a shape. The spatiotemporal perception environment representation.

[0090] The structure of the conditional diffusion Transformer decoder is as follows: Figure 5 As shown, the conditional diffusion Transformer decoder fuses obstacle geometric features with trajectory global dependency features, including the following steps:

[0091] S221. The path features output by the path encoder are embedded through position encoding to enhance the model's perception of temporal and spatial location. The time-environment fusion encoding produced by the spatiotemporal environment encoder is extracted in parallel as conditional information.

[0092] S222. After the initial diffusion trajectory noise is fused with the conditional information in the affine modulation layer, it is input into the multi-layer stacked conditional diffusion Transformer module along with the position encoding features.

[0093] S223. In each layer, the module normalizes the input features and injects conditional information through the affine modulation layer before entering the attention sub-layer—in sequence performing connection-gated attention, multi-head self-attention, and multi-head cross-attention. After each sub-module outputs, it first performs affine modulation and then adds the residual to the corresponding input.

[0094] S224. Complete attention processing, normalize, and introduce conditions through an affine modulation layer. Then, feed the data into a point-based feedforward network for high-dimensional feature mapping. The feedforward output also undergoes the same process and is added to the feedforward input residual. The attention and feedforward stages are stacked alternately in each layer. The decoder continuously fuses the trajectory and spatiotemporal conditions and iteratively predicts the path of the next time series.

[0095] S3. Apply Gaussian noise to the reference path and use a Conditional Diffusion Transformer prediction network for progressive denoising training, enabling the network to learn the denoising distribution at each diffusion step. Specifically:

[0096] The dataset is processed using a standard forward diffusion process to progressively add noise to the original path. The noise addition formula is as follows:

[0097] ;

[0098] in, Represents the original path. Representative process Noise path at each time step This represents a pre-defined noise ratio, which is cosinely increased from a smaller value to a slightly larger one.

[0099] The noise path is denoised during the training phase. diffusion time step The trajectory features generated by the path encoder and the scene embedded by the spatiotemporally aware environment encoder are input together into the denoising network. , by denoising network Predicting added noise The training objective is to minimize the mean square error between the predicted noise and the actual noise.

[0100]

[0101] The training process employs the AdamW optimizer, gradient pruning, and cosine annealing learning rate scheduling; in this embodiment, the learning rate of the AdamW optimizer is 1×10⁻⁶. -4 Weight decay 1×10 -2 The upper norm of gradient clipping is 1.0.

[0102] The training process of the Conditional Diffusion Transformer prediction network includes the following steps:

[0103] S301. Prepare training data: Path datasets generated using the RRT-Connect algorithm and optimized using the GPMP2 optimizer, along with geometric information of obstacles in the corresponding scene, are organized into training samples in batches of 16, and the coordinates and dimensions are uniformly normalized.

[0104] S302. Constructing Noise Samples: In each training iteration, the number of diffusion steps is randomly selected, and the cumulative noise weight is calculated according to the predefined noise schedule. Noisy path points are generated through the forward diffusion process.

[0105] S303, Encoding Condition Input: Input the noisy path points and environmental information into the path encoder and environment encoder respectively to obtain path features and environmental features;

[0106] S304, Denoising Network Forward: Embed the path features, environmental features, and the time step of the diffusion steps into the input conditional diffusion Transformer decoder to predict the noise residual;

[0107] S305. Calculate the loss: Evaluate the difference between the model prediction and the actual noise based on the mean square error.

[0108] S306, Gradient Backpropagation: Using the AdamW optimizer, the model parameters are updated via backpropagation under the constraint that the gradient norm does not exceed 1.0. In this example, the learning rate of the AdamW optimizer is 1×10⁻⁶. -4 Weight decay 1×10 -2 ;

[0109] S307, Training Control: The model parameters are smoothed using exponential moving average, and the learning rate is adjusted according to the cosine annealing strategy.

[0110] S308. Iteration Termination: Repeat steps S302 to S307 until the predetermined number of training steps is completed, and save the training summary and model checkpoints at the preset frequency.

[0111] S4. In each time step of the denoising training, combine the diffusion model and the cost-guided mechanism to optimize the noise path sampled from the Gaussian distribution;

[0112] The introduced cost-incentive mechanism includes collision costs. and smoothing costs Two cost functions, such as Figure 6 As shown, the mean of the backdiffusion prediction is adjusted using the following formula:

[0113] ;

[0114] ;

[0115] in, and These are coefficients representing collision cost and smoothing cost, respectively. In this example, , , The step size factor is proportional to the current noise variance.

[0116] Collision costs This is used to measure the proximity of the generated path to obstacles, and its expression is as follows:

[0117] ;

[0118] in, This indicates the position of the path at N discrete time steps. Represents the Euclidean distance between the path point and the obstacle. represent A single path point in, where, ; A preset safe distance is used; when the Euclidean distance between the path point and the obstacle is less than the safe distance, the collision cost is 0. When the distance is greater than the safe distance, the collision cost is the difference between the two distances.

[0119] Smoothing costs The expression used to constrain the second-order smoothness of the path is as follows:

[0120] ;

[0121] It is used to penalize abrupt changes between adjacent points and promote trajectory smoothing.

[0122] S5. Repeat S4 until a smooth, collision-free path is obtained.

[0123] S6. The path with the shortest smoothness and path length is selected as the optimal trajectory. The resulting multi-candidate paths are then subjected to multi-objective evaluation and inverse kinematics transformation to generate joint control commands for the robotic arm, such as... Figure 7 As shown.

[0124] S7: The robotic arm executes the planned trajectory and ensures safety and accuracy in dynamic environments through real-time sensing and online replanning.

[0125] Therefore, this invention provides a robotic arm path planning method based on Transformer and diffusion model. It utilizes an improved bidirectional RRT algorithm to generate smooth and feasible reference paths in 2D and 3D scenes, and performs post-processing on the paths to construct a training dataset containing obstacle and origin / endpoint information. A path encoder and a spatiotemporal awareness environment encoder are constructed to extract obstacle geometric features and trajectory global dependency features. These features are then fused using a conditional diffusion Transformer decoder to achieve multimodal path prediction. Gaussian noise is applied to the reference paths, and a conditional diffusion Transformer prediction network is used for progressive denoising training, enabling the network to learn the denoising distribution at each diffusion step. At each time step of the inference process, the diffusion model and cost-guided mechanism are combined to optimize the noisy path sampled from the Gaussian distribution, repeating this process until a smooth, collision-free path is obtained. The final generated multi-candidate paths are then subjected to multi-objective evaluation and inverse kinematics transformation to select the optimal trajectory and generate robotic arm joint control commands. The robotic arm executes the planned trajectory, and real-time perception and online replanning ensure safety and accuracy in dynamic environments. This method can organically combine environmental perception, Transformer encoding, and diffusion generation, enabling it to quickly generate multiple feasible trajectories that are both smooth and obstacle-avoiding in complex obstacle scenarios. It has good generalization ability and can adapt to different scenarios and dimensional changes.

[0126] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A robotic arm path planning method based on Transformer and diffusion models, characterized in that: Includes the following steps: S1. Randomly sample different obstacle configurations to initialize the environment and record environmental information. Use a hybrid planning strategy to generate reference paths in two-dimensional and three-dimensional scenes: First, obtain a rough initial trajectory through RRT-Connect, and then refine and smooth it with the help of the GPMP2 optimizer to build a training dataset containing obstacle and start-end information. S2. Construct a conditional diffusion Transformer prediction network. The conditional diffusion Transformer prediction network consists of three parts: a path encoder, a spatiotemporally aware environment encoder, and a conditional diffusion Transformer decoder. The path encoder and the spatiotemporally aware environment encoder are used to extract the geometric features of obstacles and the global dependency features of the trajectory. The conditional diffusion Transformer decoder is used to fuse the above features to achieve multimodal path prediction; S3. Apply Gaussian noise to the reference path and use a conditional diffusion Transformer prediction network for stepwise denoising training, so that the conditional diffusion Transformer prediction network learns the denoising distribution under each diffusion step. S4. In each time step of the denoising training, combine the diffusion model and the cost-guided mechanism to optimize the noise path sampled from the Gaussian distribution; The introduced cost-incentive mechanism includes collision costs. and smoothing costs These two cost functions are gradient-adjusted for the mean of the inverse diffusion prediction according to the following formula: ； ； in, and These are coefficients representing collision cost and smoothing cost, respectively. The step size factor is proportional to the current noise variance; S5. Repeat S4 until a smooth, collision-free path is obtained; S6. The path with the shortest smoothness and path length is selected as the optimal trajectory. The generated multi-candidate paths are evaluated by multiple objectives and transformed by inverse kinematics to generate joint control commands for the robotic arm. S7: The robotic arm executes the planned trajectory and ensures safety and accuracy in dynamic environments through real-time sensing and online replanning.

2. The robotic arm path planning method based on Transformer and diffusion model according to claim 1, characterized in that: In S2, the path encoder performs feature extraction by including the following steps: S201. Represent the input path sequence as a shape of... B × N × D The three-dimensional tensor, in which B For sample batch size, N The number of path points. D The original feature dimensions for each path point; S202. The original features of each path point are upscaled using the channel mapping module, and a multilayer perceptron structure is used to increase the dimensionality of the input from dimensionality. D Projected onto the middle channel dimension C The resulting shape is B × N The representation of ×C; S203, Input to a stacked 3-layer Mixer module, along the path point dimension of each layer. N Establish sequential relationships between path points to capture global structural features within the path; S204. The first channel-dimensional MLP module is used to fine-tune the features of each path point to enhance its local semantic expression capability. S205. Insert a regularization layer before the first and second MLP modules to enhance network stability and generalization ability; S206. Combine the information fusion results of the channel direction and the token direction by using the residual connection splicing method; S207, Embed information into the second MLP module. H The final path encoding representation is obtained from the 3D latent space, and the output is: B × N × H The path feature tensor of the shape.

3. The robotic arm path planning method based on Transformer and diffusion model according to claim 2, characterized in that: In S2, the spatiotemporal awareness environment encoder performs feature extraction including the following steps: S211. Represent the input environmental information as a shape of... The three-dimensional tensor, in which Indicates batch size. This indicates the number of environmental targets in each sample. Represents the feature dimensions of each target; S212. Map the time step corresponding to each sample to an MLP module. 3D time feature vector; S213, Embed the original features of each environmental target into another MLP module. The 3D latent space is mapped through a linear layer to... Dimension, and then transform it through a mapping layer. The features of trajectory points are obtained, resulting in a dimension of... The characteristic tensor; S214, will The feature tensors are input to the environment aggregation module, which includes sequentially connecting the dimensions from... Mapped to Linear layers, ReLU activation function, from Mapped to Linear layers, multi-head attention mechanisms, and The attention features are aggregated by a feature pooling layer, and the pooled global features are mapped back through a linear layer. Dimension A hidden layer set for the environment aggregation module; S215. The environmental aggregation result is fused with the temporal embedding feature, and the fused encoding result is output as a shape. The spatiotemporal perception environment representation.

4. The robotic arm path planning method based on Transformer and diffusion model according to claim 3, characterized in that: In S2, the conditional diffusion Transformer decoder fuses obstacle geometric features with trajectory global dependency features, including the following steps: S221. The path features output by the path encoder are embedded from bottom to top through position encoding to enhance the model’s perception of temporal and spatial location. The time-environment fusion encoding produced by the spatiotemporal perception environment encoder is extracted in parallel as conditional information. S222. After the initial diffusion trajectory noise is fused with the conditional information in the affine modulation layer, it is input into the multi-layer stacked conditional diffusion Transformer module along with the position encoding features. S223. In each layer of the Conditional Diffusion Transformer module, the Conditional Diffusion Transformer module normalizes the input features and injects conditional information through the affine modulation layer. It then enters the attention sub-layer and sequentially performs connection-gated attention, multi-head self-attention, and multi-head cross-attention. After each sub-module outputs, it first performs affine modulation and then adds the residual to the corresponding input. S224. Complete attention processing, normalize, and introduce conditions through an affine modulation layer. Then, feed the data into a point-based feedforward network for high-dimensional feature mapping. The feedforward output also undergoes the same process and is added to the feedforward input residual. The attention and feedforward stages are stacked alternately in each layer. The decoder continuously fuses the trajectory and spatiotemporal conditions to iteratively predict the path of the next time series.

5. The robotic arm path planning method based on Transformer and diffusion model according to claim 4, characterized in that: In S3, a standard forward diffusion process is used on the dataset to gradually add noise to the original path. The noise addition formula is as follows: ； in, Represents the original path. Representative process Noise path at each time step This represents a pre-defined noise ratio that gradually increases in a cosine manner.

6. The robotic arm path planning method based on Transformer and diffusion model according to claim 5, characterized in that: In S3, the noise path is used during the denoising training phase. diffusion time step The trajectory features generated by the path encoder and the scene embedded by the spatiotemporally aware environment encoder are input together into the denoising network. , by denoising network Predicting added noise The training objective is to minimize the mean square error between the predicted noise and the actual noise. ； The training process employs the AdamW optimizer, gradient pruning, and cosine annealing learning rate scheduling.

7. The robotic arm path planning method based on Transformer and diffusion model according to claim 6, characterized in that: In S3, the training process of the conditional diffusion Transformer prediction network includes the following steps: S301. Prepare training data: Path datasets generated using the RRT-Connect algorithm and optimized using the GPMP2 optimizer, along with geometric information of obstacles in the corresponding scene, are organized into training samples in batches of 16, and the coordinates and dimensions are uniformly normalized. S302. Constructing Noise Samples: In each training iteration, the number of diffusion steps is randomly selected, and the cumulative noise weight is calculated according to the predefined noise schedule. Noisy path points are generated through the forward diffusion process. S303, Encoding condition input: The noisy path points and environmental information are fed into the path encoder and the spatiotemporal awareness environment encoder respectively to obtain path features and environmental features; S304, Denoising Network Forward: Embed the path features, environmental features, and the time step of the diffusion steps into the input conditional diffusion Transformer decoder to predict the noise residual; S305. Calculate the loss: Evaluate the difference between the model prediction and the actual noise based on the mean square error. S306, Gradient Backpropagation: Using the AdamW optimizer, backpropagation is performed to update the model parameters under the constraint that the gradient norm does not exceed 1.0; S307, Training Control: The model parameters are smoothed using exponential moving average, and the learning rate is adjusted according to the cosine annealing strategy. S308. Iteration Termination: Repeat steps S302 to S307 until the predetermined number of training steps is completed, and save the training summary and model checkpoints at the preset frequency.

8. The robotic arm path planning method based on Transformer and diffusion model according to claim 7, characterized in that: In S4, the collision cost, used to measure the proximity of the generated path to obstacles, is expressed as follows: Collision Costs This is used to measure the proximity of the generated path to obstacles, and its expression is as follows: ； in, This indicates the position of the path at N discrete time steps. Represents the Euclidean distance between the path point and the obstacle. represent A single path point in, where, ; The default safety distance is set; when the Euclidean distance between the path point and the obstacle is less than the safety distance, the collision cost is 0; when the distance is greater than the safety distance, the collision cost is the difference between the two. Smoothing costs The expression used to constrain the second-order smoothness of the path is as follows: ； It is used to penalize abrupt changes between adjacent points and promote trajectory smoothing.

Images