Mechanical arm path planning method based on Transform and diffusion model

Abstract

The invention discloses a mechanical arm path planning method based on Transform and a diffusion model, and belongs to the technical field of robots, and the method comprises the following steps: collecting environment information to generate a reference path, and constructing a training data set; a conditional diffusion Transform prediction network is constructed, features are extracted, and multi-modal path prediction is realized; gaussian noise is applied to the reference path, and denoising training is carried out on the conditional diffusion Transform prediction network; in combination with a diffusion model and a cost guidance mechanism, optimizing a noise path sampled from Gaussian distribution until a smooth collision-free path is generated; the candidate paths are evaluated, and a mechanical arm joint control instruction is generated; and the mechanical arm executes the planning track and carries out real-time sensing and online re-planning. According to the method, environment perception, Transform coding and diffusion generation are organically combined, a plurality of feasible tracks which are smooth and capable of avoiding obstacles are rapidly generated in a complex obstacle scene, and the method has good generalization ability and can adapt to different scenes and dimension changes.

Description

Technical Field

[0001] This invention relates to the field of robotics, and in particular to a path planning method for a robotic arm based on the Transformer and diffusion model. Background Technology

[0002] Path planning is a key technology for achieving autonomous movement in robotic systems and has been widely applied in scenarios such as industrial manufacturing, warehousing and logistics, autonomous driving, and emergency rescue. Traditional graph search and optimization methods can generate feasible paths in static or regularized environments, but when faced with high-dimensional scenarios, they often require significant preprocessing or online replanning costs. Furthermore, because the path generation process involves cascading decisions, small errors can easily accumulate, leading to a decrease in the smoothness and feasibility of the final trajectory.

[0003] Furthermore, existing learning-based path planning algorithms typically rely on fixed scenario data during model training, resulting in limited generalization ability and insufficient rapid adaptability to new environments. This makes it difficult to meet the dual requirements of real-time performance and robustness in complex and ever-changing production or operational environments.

[0004] The recently emerging diffusion model offers a new approach to generative modeling. It simulates data distribution through progressive denoising, directly generating complete trajectory sequences in a high-dimensional continuous space, and naturally possesses smooth and coherent characteristics. Introducing such models into the field of path planning holds promise for overcoming the bottlenecks of traditional methods in long-term dependencies and error accumulation.

[0005] However, in the implementation process, key issues such as the effective expression of environmental constraints, efficient sampling of high-dimensional action spaces, and precise guidance of target conditions need to be considered. In particular, in practical deployment, how to maintain model generation efficiency while taking into account environmental adaptability and result controllability is a core challenge that current research urgently needs to address.

[0006] Therefore, there is an urgent need for a novel framework that can capture the spatiotemporal characteristics of the environment and efficiently generate globally optimal or near-optimal paths under target guidance, in order to improve the planning accuracy and execution reliability of robots in complex scenarios. This framework should not only support multidimensional representations of obstacle distribution and constraints, but also have the ability to conditionally modulate the generated results, thereby achieving efficient and robust path generation and online adjustment in uncertain environments. Summary of the Invention

[0007] The purpose of this invention is to provide a robotic arm path planning method based on Transformer and diffusion model, which can organically combine environmental perception, Transformer encoding and diffusion generation, and can quickly generate multiple feasible trajectories that are both smooth and obstacle-avoiding in complex obstacle scenarios. It has good generalization ability and can adapt to different scenarios and dimensional changes.

[0008] To achieve the above objectives, this invention provides a robotic arm path planning method based on the Transformer and diffusion model, comprising the following steps:

[0009] S1. Randomly sample different obstacle configurations to initialize the environment and record environmental information. Use a hybrid planning strategy to generate reference paths in two-dimensional and three-dimensional scenes: First, obtain a rough initial trajectory through RRT-Connect, and then refine and smooth it with the help of the GPMP2 optimizer to build a training dataset containing obstacle and start-end information.

[0010] S2. Construct a conditional diffusion Transformer prediction network, which consists of three parts: a path encoder, a spatiotemporal awareness environment encoder, and a conditional diffusion Transformer decoder. The path encoder and spatiotemporal awareness environment encoder are used to extract obstacle geometric features and trajectory global dependency features. The conditional diffusion Transformer decoder is used to fuse the above features to achieve multimodal path prediction.

[0011] S3. Apply Gaussian noise to the reference path and use a conditional diffusion Transformer prediction network for stepwise denoising training, so that the conditional diffusion Transformer prediction network learns the denoising distribution under each diffusion step.

[0012] S4. In each time step of the denoising training, combine the diffusion model and the cost-guided mechanism to optimize the noise path sampled from the Gaussian distribution;

[0013] S5. Repeat S4 until a smooth, collision-free path is obtained;

[0014] S6. The path with the shortest smoothness and path length is selected as the optimal trajectory. The generated multi-candidate paths are evaluated by multiple objectives and transformed by inverse kinematics to generate joint control commands for the robotic arm.

[0015] S7: The robotic arm executes the planned trajectory and ensures safety and accuracy in dynamic environments through real-time sensing and online replanning.

[0016] Preferably, in S2, the path encoder performs feature extraction by including the following steps:

[0017] S201. Represent the input path sequence as a three-dimensional tensor of shape B×N×D, where B is the sample batch size, N is the number of path points, and D is the original feature dimension of each path point.

[0018] S202. The original features of each path point are upgraded by the channel mapping module. The input is projected from dimension D to the intermediate channel dimension C using a multilayer perceptron structure to obtain a representation with shape B×N×C.

[0019] S203. Input into a stacked 3-layer Mixer module, establish sequential associations between path points along the path point dimension N in each layer to capture global structural features in the path;

[0020] S204. The first channel-dimensional MLP module is used to fine-tune the features of each path point to enhance its local semantic expression capability.

[0021] S205. Insert a regularization layer before the first and second MLP modules to enhance network stability and generalization ability;

[0022] S206. Combine the information fusion results of the channel direction and the token direction by using the residual connection splicing method;

[0023] S207. The information is embedded into the H-dimensional latent space through the second MLP module to obtain the final path encoding representation, and the output is a path feature tensor of shape B×N×H.

[0024] Preferably, in S2, the spatiotemporal awareness environment encoder performs feature extraction by including the following steps:

[0025] S211. Represent the input environmental information as a three-dimensional tensor of shape B×N0×D0, where B represents the batch size, N0 represents the number of environmental targets in each sample, and D0 represents the feature dimension of each target.

[0026] S212. Map the time step corresponding to each sample to an H-dimensional time feature vector through an MLP module;

[0027] S213. The original features of each environmental target are embedded into the H-dimensional latent space through another MLP module. Then, it is mapped to D0 dimensions through a linear layer, and then transformed into N trajectory point features through a mapping layer to obtain a feature tensor with dimensions B×N×H.

[0028] S214. Input the feature tensor of B×N×H into the environment aggregation module. The environment aggregation module includes a linear layer that maps the dimension from H to D, a ReLU activation function, a linear layer that maps from D to E, a multi-head attention mechanism, a feature pooling layer that aggregates the attention features of B×N×E, and maps the pooled global features back to the H dimension through the linear layer. E is the hidden layer set by the environment aggregation module.

[0029] S215. The environment aggregation result is fused with the temporal embedding feature, and the fused encoding result is output as a spatiotemporally aware environment representation of shape B×H.

[0030] Preferably, in S2, the conditional diffusion Transformer decoder fuses obstacle geometric features and trajectory global dependency features, including the following steps:

[0031] S221. The path features output by the path encoder are embedded from bottom to top through position encoding to enhance the model’s perception of temporal and spatial location. The time-environment fusion encoding produced by the spatiotemporal perception environment encoder is extracted in parallel as conditional information.

[0032] S222. After the initial diffusion trajectory noise is fused with the conditional information in the affine modulation layer, it is input into the multi-layer stacked conditional diffusion Transformer module along with the position encoding features.

[0033] S223. In each layer of the Conditional Diffusion Transformer module, the Conditional Diffusion Transformer module normalizes the input features and injects conditional information through the affine modulation layer. It then enters the attention sub-layer and sequentially performs connection-gated attention, multi-head self-attention, and multi-head cross-attention. After each sub-module outputs, it first performs affine modulation and then adds the residual to the corresponding input.

[0034] S224. Complete attention processing, normalize, and introduce conditions through an affine modulation layer. Then, feed the data into a point-based feedforward network for high-dimensional feature mapping. The feedforward output also undergoes the same process and is added to the feedforward input residual. The attention and feedforward stages are stacked alternately in each layer. The decoder continuously fuses the trajectory and spatiotemporal conditions to iteratively predict the path of the next time series.

[0035] Preferably, in S3, a standard forward diffusion process is used on the dataset to progressively add noise to the original path. The noise addition formula is as follows:

[0036]

[0037] Where w0 represents the original path, w t β represents the noise path that has passed through t time steps. t ∈(0,1) represents a pre-defined noise ratio that gradually increases in the form of a cosine.

[0038] Preferably, in S3, the noise path w is used in the denoising training phase. t The diffusion time step t, the trajectory features generated by the path encoder, and the scene embedding generated by the spatiotemporally aware environment encoder are all input into the denoising network. θ , by denoising network ∈ θ Predicting added noise The training objective is to minimize the mean square error between the predicted noise and the actual noise.

[0039]

[0040] The training process employs the AdamW optimizer, gradient pruning, and cosine annealing learning rate scheduling.

[0041] Preferably, in S3, the training process of the conditional diffusion Transformer prediction network includes the following steps:

[0042] S301. Prepare training data: Path datasets generated using the RRT-Connect algorithm and optimized using the GPMP2 optimizer, along with geometric information of obstacles in the corresponding scene, are organized into training samples in batches of 16, and the coordinates and dimensions are uniformly normalized.

[0043] S302. Constructing Noise Samples: In each training iteration, the number of diffusion steps is randomly selected, and the cumulative noise weight is calculated according to the predefined noise schedule. Noisy path points are generated through the forward diffusion process.

[0044] S303, Encoding condition input: The noisy path points and environmental information are fed into the path encoder and the spatiotemporal awareness environment encoder respectively to obtain path features and environmental features;

[0045] S304, Denoising Network Forward: Embed the path features, environmental features, and the time step of the diffusion steps into the input conditional diffusion Transformer decoder to predict the noise residual;

[0046] S305. Calculate the loss: Evaluate the difference between the model prediction and the actual noise based on the mean square error.

[0047] S306, Gradient Backpropagation: Using the AdamW optimizer, backpropagation is performed to update the model parameters under the constraint that the gradient norm does not exceed 1.0;

[0048] S307, Training Control: The model parameters are smoothed using exponential moving average, and the learning rate is adjusted according to the cosine annealing strategy.

[0049] S308. Iteration Termination: Repeat steps S302 to S307 until the predetermined number of training steps is completed, and save the training summary and model checkpoints at the preset frequency.

[0050] Preferably, in S4, the introduced cost guidance mechanism includes two cost functions: collision cost and smoothing cost. The mean of the backdiffusion prediction is adjusted by gradient according to the following formula:

[0051]

[0052] w t-1 =w t-1 '+ηg;

[0053] Where, λ c With λ s These represent the coefficients for collision cost and smoothing cost, respectively, and η is the step size coefficient proportional to the current noise variance.

[0054] Preferably, in S4, the collision cost, used to measure the proximity of the generated path to obstacles, is expressed as follows:

[0055] Collision cost C c This is used to measure the proximity of the generated path to obstacles, and its expression is as follows:

[0056]

[0057] Where w = {p1,…,p} N} represents the position of the path at N discrete time steps. d represents the Euclidean distance between the trajectory point and the obstacle. s The default safety distance is set; when the Euclidean distance between the path point and the obstacle is less than the safety distance, the collision cost is 0; when the distance is greater than the safety distance, the collision cost is the difference between the two.

[0058] Smoothing cost C s The expression used to constrain the second-order smoothness of the path is as follows:

[0059]

[0060] It is used to penalize abrupt changes between adjacent points and promote trajectory smoothing.

[0061] Therefore, this invention provides a robotic arm path planning method based on Transformer and diffusion model. It utilizes the RRT-Connect algorithm and GPMP2 optimizer to generate smooth and feasible reference paths in 2D and 3D scenes, and performs post-processing on the paths to construct a training dataset containing obstacle and origin / endpoint information. A path encoder and a spatiotemporal awareness environment encoder are constructed to extract obstacle geometric features and trajectory global dependency features. These features are then fused using a conditional diffusion Transformer decoder to achieve multimodal path prediction. Gaussian noise is applied to the reference paths, and a conditional diffusion Transformer prediction network is used for progressive denoising training, enabling the network to learn the denoising distribution at each diffusion step. At each time step of the inference process, the diffusion model and cost-guided mechanism are combined to optimize the noisy path sampled from the Gaussian distribution, repeating this process until a smooth, collision-free path is obtained. The final generated multi-candidate paths are then subjected to multi-objective evaluation and inverse kinematics transformation to select the optimal trajectory and generate robotic arm joint control commands. The robotic arm executes the planned trajectory, and real-time perception and online replanning ensure safety and accuracy in dynamic environments. This method can organically combine environmental perception, Transformer encoding, and diffusion generation, enabling it to quickly generate multiple feasible trajectories that are both smooth and obstacle-avoiding in complex obstacle scenarios. It has good generalization ability and can adapt to different scenarios and dimensional changes.

[0062] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0063] Figure 1 This is an overall flowchart of a robotic arm path planning method based on Transformer and diffusion model according to the present invention;

[0064] Figure 2 This is a schematic diagram of the dataset configuration of this invention;

[0065] Figure 3 This is a schematic diagram of feature extraction performed by the path encoder of the present invention;

[0066] Figure 4 This is a structural diagram of the spatiotemporal sensing environment encoder of the present invention;

[0067] Figure 5 This is a schematic diagram of the conditional diffusion Transformer decoder structure of the present invention;

[0068] Figure 6 This is a schematic diagram of the conditional guidance structure of the present invention;

[0069] Figure 7 This is a schematic diagram of a robotic arm path planning method based on Transformer and diffusion model according to the present invention. Detailed Implementation

[0070] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0071] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0072] Example 1

[0073] This invention provides a robotic arm path planning method based on the Transformer and diffusion model, the overall flowchart of which is shown below. Figure 1 As shown, the specific process includes the following steps:

[0074] S1. Randomly sample different obstacle configurations to initialize the environment and record environmental information. A hybrid planning strategy is used to generate reference paths in both 2D and 3D scenes: First, a coarse initial trajectory is obtained through RRT-Connect, then refined and smoothed using the GPMP2 optimizer to construct a training dataset containing obstacle and start / endpoint information. To ensure the diversity and richness of the training data, this embodiment randomly generates 1000 independent scenes, sets 100 sets of start / endpoints as hard constraints in each scene, and plans 10 paths for each configuration, thus providing sufficient and reliable training samples for the diffusion model. The training samples are as follows: Figure 2 As shown.

[0075] S2. Construct a conditional diffusion Transformer prediction network, which consists of three parts: a path encoder, a spatiotemporal awareness environment encoder, and a conditional diffusion Transformer decoder. The path encoder and spatiotemporal awareness environment encoder are used to extract obstacle geometric features and trajectory global dependency features. The conditional diffusion Transformer decoder is used to fuse the above features to achieve multimodal path prediction.

[0076] Path encoder performs feature extraction, such as Figure 3As shown, it includes the following steps:

[0077] S201. Represent the input path sequence as a three-dimensional tensor of shape B×N×D, where B is the sample batch size, N is the number of path points, and D is the original feature dimension of each path point.

[0078] S202. The original features of each path point are upgraded by the channel mapping module. The input is projected from dimension D to the intermediate channel dimension C using a multilayer perceptron structure to obtain a representation with shape B×N×C.

[0079] S203. Input into a stacked 3-layer Mixer module, establish sequential associations between path points along the path point dimension N in each layer to capture global structural features in the path;

[0080] S204. The first channel-dimensional MLP module is used to fine-tune the features of each path point to enhance its local semantic expression capability.

[0081] S205. Insert a regularization layer before the first and second MLP modules to enhance network stability and generalization ability;

[0082] S206. Combine the information fusion results of the channel direction and the token direction by using the residual connection splicing method;

[0083] S207. The information is embedded into the H-dimensional latent space through the second MLP to obtain the final path encoding representation, and the output is a path feature tensor of shape B×N×H.

[0084] Spatiotemporal awareness environment encoders, such as Figure 4 As shown, feature extraction includes the following steps:

[0085] S211. Represent the input environmental information as a three-dimensional tensor of shape B×N0×D0, where B represents the batch size, N0 represents the number of environmental targets in each sample, and D0 represents the feature dimension of each target.

[0086] S212. Map the time step corresponding to each sample to an H-dimensional time feature vector through an MLP module;

[0087] S213. The original features of each environmental target are embedded into the H-dimensional latent space through another MLP module. Then, it is mapped to D0 dimensions through a linear layer, and then transformed into N trajectory point features through a mapping layer to obtain a feature tensor with dimensions B×N×H.

[0088] S214. The feature is input to the environment aggregation module. The module includes a linear layer that maps the dimension from H to D, a ReLU activation function, a linear layer that maps from D to E, a multi-head attention mechanism, and a feature pooling layer that aggregates the attention features of B×N×E. The pooled global features are then mapped back to the H dimension through the linear layer. E is the hidden layer set by the environment aggregation module.

[0089] S215. The environment aggregation result is fused with the temporal embedding feature, and the fused encoding result is output as a spatiotemporally aware environment representation of shape B×H.

[0090] The structure of the conditional diffusion Transformer decoder is as follows: Figure 5 As shown, the conditional diffusion Transformer decoder fuses obstacle geometric features with trajectory global dependency features, including the following steps:

[0091] S221. The path features output by the path encoder are embedded through position encoding to enhance the model's perception of temporal and spatial location. The time-environment fusion encoding produced by the spatiotemporal environment encoder is extracted in parallel as conditional information.

[0092] S222. After the initial diffusion trajectory noise is fused with the conditional information in the affine modulation layer, it is input into the multi-layer stacked conditional diffusion Transformer module along with the position encoding features.

[0093] S223. In each layer, the module normalizes the input features and injects conditional information through the affine modulation layer before entering the attention sub-layer—in sequence performing connection-gated attention, multi-head self-attention, and multi-head cross-attention. After each sub-module outputs, it first performs affine modulation and then adds the residual to the corresponding input.

[0094] S224. Complete attention processing, normalize, and introduce conditions through an affine modulation layer. Then, feed the data into a point-based feedforward network for high-dimensional feature mapping. The feedforward output also undergoes the same process and is added to the feedforward input residual. The attention and feedforward stages are stacked alternately in each layer. The decoder continuously fuses the trajectory and spatiotemporal conditions and iteratively predicts the path of the next time series.

[0095] S3. Apply Gaussian noise to the reference path and use a Conditional Diffusion Transformer prediction network for progressive denoising training, enabling the network to learn the denoising distribution at each diffusion step. Specifically:

[0096] The dataset is processed using a standard forward diffusion process to progressively add noise to the original path. The noise addition formula is as follows:

[0097]

[0098] Where w0 represents the original path, w t β represents the noise path that has passed through t time steps. t ∈(0,1) represents a pre-defined noise ratio, which increases cosinely from a small value to a slightly larger value.

[0099] During the denoising training phase, the noise path w t The diffusion time step t, the trajectory features generated by the path encoder, and the scene embedding generated by the spatiotemporally aware environment encoder are all input into the denoising network. θ , by denoising network ∈ θ Predicting added noise The training objective is to minimize the mean square error between the predicted noise and the actual noise.

[0100]

[0101] The training process employs the AdamW optimizer, gradient pruning, and cosine annealing learning rate scheduling; in this embodiment, the learning rate of the AdamW optimizer is 1×10⁻⁶. -4 Weight decay 1×10 -2 The upper norm of gradient clipping is 1.0.

[0102] The training process of the Conditional Diffusion Transformer prediction network includes the following steps:

[0103] S301. Prepare training data: Path datasets generated using the RRT-Connect algorithm and optimized using the GPMP2 optimizer, along with geometric information of obstacles in the corresponding scene, are organized into training samples in batches of 16, and the coordinates and dimensions are uniformly normalized.

[0104] S302. Constructing Noise Samples: In each training iteration, the number of diffusion steps is randomly selected, and the cumulative noise weight is calculated according to the predefined noise schedule. Noisy path points are generated through the forward diffusion process.

[0105] S303, Encoding Condition Input: Input the noisy path points and environmental information into the path encoder and environment encoder respectively to obtain path features and environmental features;

[0106] S304, Denoising Network Forward: Embed the path features, environmental features, and the time step of the diffusion steps into the input conditional diffusion Transformer decoder to predict the noise residual;

[0107] S305. Calculate the loss: Evaluate the difference between the model prediction and the actual noise based on the mean square error.

[0108] S306, Gradient Backpropagation: Using the AdamW optimizer, the model parameters are updated via backpropagation under the constraint that the gradient norm does not exceed 1.0. In this example, the learning rate of the AdamW optimizer is 1×10⁻⁶. -4 Weight decay 1×10 -2 ;

[0109] S307, Training Control: The model parameters are smoothed using exponential moving average, and the learning rate is adjusted according to the cosine annealing strategy.

[0110] S308. Iteration Termination: Repeat steps S302 to S307 until the predetermined number of training steps is completed, and save the training summary and model checkpoints at the preset frequency.

[0111] S4. In each time step of the denoising training, combine the diffusion model and the cost-guided mechanism to optimize the noise path sampled from the Gaussian distribution;

[0112] The introduced cost-guiding mechanism includes two cost functions: collision cost and smoothing cost, such as Figure 6 As shown, the mean of the backdiffusion prediction is adjusted using the following formula:

[0113]

[0114] w t-1 =w t-1 '+ηg;

[0115] Where, λ c With λ s λ represents the coefficients for collision cost and smoothing cost, respectively. In this example, λ c =0.01, λ s =1×10 -8 η is a step size coefficient that is proportional to the current noise variance.

[0116] Collision cost C c This is used to measure the proximity of the generated path to obstacles, and its expression is as follows:

[0117]

[0118] Where w = {p1,…,p} N} represents the position of the path at N discrete time steps. d represents the Euclidean distance between the trajectory point and the obstacle. s A preset safe distance is used; when the Euclidean distance between the path point and the obstacle is less than the safe distance, the collision cost is 0. When the distance is greater than the safe distance, the collision cost is the difference between the two distances.

[0119] Smoothing cost C sThe expression used to constrain the second-order smoothness of the path is as follows:

[0120]

[0121] It is used to penalize abrupt changes between adjacent points and promote trajectory smoothing.

[0122] S5. Repeat S4 until a smooth, collision-free path is obtained.

[0123] S6. The path with the shortest smoothness and path length is selected as the optimal trajectory. The resulting multi-candidate paths are then subjected to multi-objective evaluation and inverse kinematics transformation to generate joint control commands for the robotic arm, such as... Figure 7 As shown.

[0124] S7: The robotic arm executes the planned trajectory and ensures safety and accuracy in dynamic environments through real-time sensing and online replanning.

[0125] Therefore, this invention provides a robotic arm path planning method based on Transformer and diffusion model. It utilizes an improved bidirectional RRT algorithm to generate smooth and feasible reference paths in 2D and 3D scenes, and performs post-processing on the paths to construct a training dataset containing obstacle and origin / endpoint information. A path encoder and a spatiotemporal awareness environment encoder are constructed to extract obstacle geometric features and trajectory global dependency features. These features are then fused using a conditional diffusion Transformer decoder to achieve multimodal path prediction. Gaussian noise is applied to the reference paths, and a conditional diffusion Transformer prediction network is used for progressive denoising training, enabling the network to learn the denoising distribution at each diffusion step. At each time step of the inference process, the diffusion model and cost-guided mechanism are combined to optimize the noisy path sampled from the Gaussian distribution, repeating this process until a smooth, collision-free path is obtained. The final generated multi-candidate paths are then subjected to multi-objective evaluation and inverse kinematics transformation to select the optimal trajectory and generate robotic arm joint control commands. The robotic arm executes the planned trajectory, and real-time perception and online replanning ensure safety and accuracy in dynamic environments. This method can organically combine environmental perception, Transformer encoding, and diffusion generation, enabling it to quickly generate multiple feasible trajectories that are both smooth and obstacle-avoiding in complex obstacle scenarios. It has good generalization ability and can adapt to different scenarios and dimensional changes.

[0126] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A robotic arm path planning method based on Transformer and diffusion models, characterized in that: Includes the following steps: S1. Randomly sample different obstacle configurations to initialize the environment and record environmental information. Use a hybrid planning strategy to generate reference paths in two-dimensional and three-dimensional scenes: First, obtain a rough initial trajectory through RRT-Connect, and then refine and smooth it with the help of the GPMP2 optimizer to build a training dataset containing obstacle and start-end information. S2. Construct a conditional diffusion Transformer prediction network. The conditional diffusion Transformer prediction network consists of three parts: a path encoder, a spatiotemporally aware environment encoder, and a conditional diffusion Transformer decoder. The path encoder and the spatiotemporally aware environment encoder are used to extract the geometric features of obstacles and the global dependency features of the trajectory. The conditional diffusion Transformer decoder is used to fuse the above features to achieve multimodal path prediction; S3. Apply Gaussian noise to the reference path and use a conditional diffusion Transformer prediction network for stepwise denoising training, so that the conditional diffusion Transformer prediction network learns the denoising distribution under each diffusion step. S4. In each time step of the denoising training, combine the diffusion model and the cost-guided mechanism to optimize the noise path sampled from the Gaussian distribution; S5. Repeat S4 until a smooth, collision-free path is obtained; S6. The path with the shortest smoothness and path length is selected as the optimal trajectory. The generated multi-candidate paths are evaluated by multiple objectives and transformed by inverse kinematics to generate joint control commands for the robotic arm. S7: The robotic arm executes the planned trajectory and ensures safety and accuracy in dynamic environments through real-time sensing and online replanning.

2. The robotic arm path planning method based on Transformer and diffusion model according to claim 1, characterized in that: In S2, the path encoder performs feature extraction by including the following steps: S201. Represent the input path sequence as a three-dimensional tensor of shape B×N×D, where B is the sample batch size, N is the number of path points, and D is the original feature dimension of each path point. S202. The original features of each path point are upgraded by the channel mapping module. The input is projected from dimension D to the intermediate channel dimension C by a multilayer perceptron structure to obtain a representation with shape B×N×C. S203. Input into a stacked 3-layer Mixer module, establish sequential associations between path points along the path point dimension N in each layer to capture global structural features in the path; S204. The first channel-dimensional MLP module is used to fine-tune the features of each path point to enhance its local semantic expression capability. S205. Insert a regularization layer before the first and second MLP modules to enhance network stability and generalization ability. S206. Combine the information fusion results of the channel direction and the token direction by using the residual connection splicing method; S207. The information is embedded into the H-dimensional latent space through the second MLP module to obtain the final path encoding representation, and the output is a path feature tensor of shape B×N×H.

3. The robotic arm path planning method based on Transformer and diffusion model according to claim 2, characterized in that: In S2, the spatiotemporal awareness environment encoder performs feature extraction including the following steps: S211. Represent the input environmental information as a three-dimensional tensor of shape B×N0×D0, where B represents the batch size, N0 represents the number of environmental targets in each sample, and D0 represents the feature dimension of each target. S212. Map the time step corresponding to each sample to an H-dimensional time feature vector through an MLP module; S213. The original features of each environmental target are embedded into the H-dimensional latent space through another MLP module, mapped to D0 dimensions through a linear layer, and then transformed into N trajectory point features through a mapping layer to obtain a feature tensor with dimensions B×N×H. S214. Input the feature tensor of B×N×H into the environment aggregation module. The environment aggregation module includes a linear layer that maps the dimension from H to D, a ReLU activation function, a linear layer that maps from D to E, a multi-head attention mechanism, and a feature pooling layer that aggregates the attention features of B×N×E. The pooled global features are then mapped back to the H dimension through the linear layer. E is the hidden layer set by the environment aggregation module. S215. The environment aggregation result is fused with the temporal embedding feature, and the fused encoding result is output as a spatiotemporally aware environment representation of shape B×H.

4. The robotic arm path planning method based on Transformer and diffusion model according to claim 3, characterized in that: In S2, the conditional diffusion Transformer decoder fuses obstacle geometric features with trajectory global dependency features, including the following steps: S221. The path features output by the path encoder are embedded from bottom to top through position encoding to enhance the model’s perception of temporal and spatial location. The time-environment fusion encoding produced by the spatiotemporal perception environment encoder is extracted in parallel as conditional information. S222. After the initial diffusion trajectory noise is fused with the conditional information in the affine modulation layer, it is input into the multi-layer stacked conditional diffusion Transformer module along with the position encoding features. S223. In each layer of the Conditional Diffusion Transformer module, the Conditional Diffusion Transformer module normalizes the input features and injects conditional information through the affine modulation layer. It then enters the attention sub-layer and sequentially performs connection-gated attention, multi-head self-attention, and multi-head cross-attention. After each sub-module outputs, it first performs affine modulation and then adds the residual to the corresponding input. S224. Complete attention processing, normalize, and introduce conditions through an affine modulation layer. Then, feed the data into a point-based feedforward network for high-dimensional feature mapping. The feedforward output also undergoes the same process and is added to the feedforward input residual. The attention and feedforward stages are stacked alternately in each layer. The decoder continuously fuses the trajectory and spatiotemporal conditions to iteratively predict the path of the next time series.

5. The robotic arm path planning method based on Transformer and diffusion model according to claim 4, characterized in that: In S3, a standard forward diffusion process is used on the dataset to gradually add noise to the original path. The noise addition formula is as follows: Where w0 represents the original path, w t β represents the noise path that has passed through t time steps. t ∈(0,1) represents a pre-defined noise ratio that gradually increases in the form of a cosine.

6. The robotic arm path planning method based on Transformer and diffusion model according to claim 5, characterized in that: In S3, the noise path w is used in the denoising training phase. t The diffusion time step t, the trajectory features generated by the path encoder, and the scene embedding generated by the spatiotemporally aware environment encoder are all input into the denoising network. θ , by denoising network ∈ θ Predicting added noise The training objective is to minimize the mean square error between the predicted noise and the actual noise. The training process employs the AdamW optimizer, gradient pruning, and cosine annealing learning rate scheduling.

7. The robotic arm path planning method based on Transformer and diffusion model according to claim 6, characterized in that: In S3, the training process of the conditional diffusion Transformer prediction network includes the following steps: S301. Prepare training data: Path datasets generated using the RRT-Connect algorithm and optimized using the GPMP2 optimizer, along with geometric information of obstacles in the corresponding scene, are organized into training samples in batches of 16, and the coordinates and dimensions are uniformly normalized. S302. Constructing Noise Samples: In each training iteration, the number of diffusion steps is randomly selected, and the cumulative noise weight is calculated according to the predefined noise schedule. Noisy path points are generated through the forward diffusion process. S303, Encoding condition input: The noisy path points and environmental information are fed into the path encoder and the spatiotemporal awareness environment encoder respectively to obtain path features and environmental features; S304, Denoising Network Forward: Embed the path features, environmental features, and the time step of the diffusion steps into the input conditional diffusion Transformer decoder to predict the noise residual; S305. Calculate the loss: Evaluate the difference between the model prediction and the actual noise based on the mean square error. S306, Gradient Backpropagation: Using the AdamW optimizer, backpropagation is performed to update the model parameters under the constraint that the gradient norm does not exceed 1.0; S307, Training Control: The model parameters are smoothed using exponential moving average, and the learning rate is adjusted according to the cosine annealing strategy. S308. Iteration Termination: Repeat steps S302 to S307 until the predetermined number of training steps is completed, and save the training summary and model checkpoints at the preset frequency.

8. The robotic arm path planning method based on Transformer and diffusion model according to claim 7, characterized in that: In S4, the introduced cost guidance mechanism includes two cost functions: collision cost and smoothing cost. The gradient adjustment of the inverse diffusion prediction mean is performed according to the following formula: In t-1 =in t-1 '+ηg; Where, λ c With λ s These represent the coefficients for collision cost and smoothing cost, respectively, and η is the step size coefficient proportional to the current noise variance.

9. The robotic arm path planning method based on Transformer and diffusion model according to claim 8, characterized in that: In S4, the collision cost, used to measure the proximity of the generated path to obstacles, is expressed as follows: Collision cost C c This is used to measure the proximity of the generated path to obstacles, and its expression is as follows: Where w = {p1,…,p} N } represents the position of the path at N discrete time steps. d represents the Euclidean distance between the trajectory point and the obstacle. s The default safety distance is set; when the Euclidean distance between the path point and the obstacle is less than the safety distance, the collision cost is 0; when the distance is greater than the safety distance, the collision cost is the difference between the two. Smoothing cost C s The expression used to constrain the second-order smoothness of the path is as follows: It is used to penalize abrupt changes between adjacent points and promote trajectory smoothing.

Images