Trajectory generation method and system based on hierarchical GRPO truncated diffusion model

Abstract

The application provides a hierarchical GRPO-based truncated diffusion model trajectory generation method and system. The method obtains the current environment features around an autonomous vehicle, generates multi-anchor candidate trajectories with the current environment features as anchor trajectory generation conditions, superimposes the multi-anchor candidate trajectories with first multiplicative noise, performs smoothing filtering on the multi-anchor exploration trajectories, calculates the intra-group reward scores of each trajectory variant in each trajectory variant group, takes the trajectory variant with the maximum intra-group reward score in each trajectory variant group as a screening trajectory variant, performs global safety evaluation on each screening trajectory variant, corrects the intra-group reward scores based on the global safety evaluation results for each screening trajectory variant, performs global safety evaluation on the smoothed exploration trajectories, and screens all screening trajectory variants and smoothed exploration trajectories. The application can effectively constrain low-quality patterns and explore better strategies while maintaining trajectory diversity.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a method and system for generating truncation diffusion model trajectories based on hierarchical GRPO. Background Technology

[0002] In autonomous driving, planning algorithms need to handle the uncertainty of future scenarios and generate diverse trajectories. Diffusion models have been introduced due to their powerful multimodal generation capabilities. However, existing technologies such as DiffusionDrive, although they avoid mode collapse by dividing driving intentions through anchors, still face two key problems: (1) The dilemma of quality and diversity: These models are mainly based on imitation learning training, and only supervise the "positive anchors" that are closest to the true trajectory, while lacking constraints on the trajectories generated by other "negative anchors". This results in the model being able to generate diverse trajectories, but also accompanied by a large number of low-quality or even collision-causing invalid trajectories. (2) The problem of reinforcement learning adaptation: Reinforcement learning (such as GRPO (Group Relative Policy Optimization)) can effectively improve model performance, but if it is directly applied to anchor diffusion models to compare the merits of trajectories for different driving intentions (such as left turn and straight), it will force the model to collapse into a single mode, stifling its inherent diversity.

[0003] Therefore, there is an urgent need for a method that can effectively constrain low-quality patterns and explore better strategies while maintaining trajectory diversity. Summary of the Invention

[0004] In view of the shortcomings of the prior art described above, this application provides a method and system for generating truncation diffusion model trajectories based on hierarchical GRPO to solve the above-mentioned technical problems.

[0005] According to one aspect of the embodiments of this application, a trajectory generation method based on a truncated diffusion model using a hierarchical GRPO is provided. The trajectory generation method includes: acquiring current environmental features around an autonomous vehicle; the current environmental features are obtained by encoding and fusing current environmental information; the current environmental information includes image information and map information; using the current environmental features as anchor point trajectory generation conditions, generating multi-anchor point candidate trajectories through a basic trajectory generator; superimposing the multi-anchor point candidate trajectories with a first multiplicative noise to obtain a multi-anchor point exploration trajectory; and performing smoothing filtering on the multi-anchor point exploration trajectory to obtain a smooth exploration trajectory; sampling the smooth exploration trajectory according to a first preset anchor point to obtain multiple trajectory variant groups; and calculating each trajectory variant within each trajectory variant group. The system calculates the intra-group reward score for each trajectory variant; selects the trajectory variant with the highest intra-group reward score within each trajectory variant group as the selected trajectory variant; performs a global safety assessment on each selected trajectory variant based on the truncated GRPO model between anchor points, obtaining the global safety assessment result for each selected trajectory variant; for each selected trajectory variant, corrects the intra-group reward score based on the global safety assessment result, obtaining the global reward score for each selected trajectory variant; and performs a global safety assessment on the smooth exploration trajectory based on the truncated GRPO model between anchor points, obtaining the global reward score for the smooth exploration trajectory; based on the global reward scores of each selected trajectory variant and the smooth exploration trajectory, filters all selected trajectory variants and the smooth exploration trajectory to obtain the optimal driving trajectory.

[0006] In one embodiment of this application, the process of calculating the intra-group reward score of each trajectory variant within each trajectory variant group includes: obtaining the efficiency reward, comfort reward, and rule compliance reward of each trajectory variant within each trajectory variant group; performing a linear weighted summation of the efficiency reward, comfort reward, and rule compliance reward of each trajectory variant according to each trajectory variant group to obtain the comprehensive reward score of each trajectory variant within each trajectory variant group; calculating the mean and standard deviation of the comprehensive reward score of each trajectory variant within each trajectory variant group based on the comprehensive reward score of each trajectory variant within each trajectory variant group; and normalizing the comprehensive reward score of each trajectory variant within the corresponding group based on the mean and standard deviation of the comprehensive reward score of each trajectory variant within each trajectory variant group to obtain the intra-group reward score of each trajectory variant within each trajectory variant group.

[0007] In one embodiment of this application, for each selected trajectory variant, if the global safety assessment result indicates a collision, the intra-group reward score is corrected to a preset negative value; if the global safety assessment result indicates no collision and the intra-group reward score is less than a preset score threshold, the intra-group reward score is corrected to the preset score threshold; if the global safety assessment result indicates no collision and the intra-group reward score is greater than or equal to the preset score threshold, the intra-group reward score is not corrected.

[0008] For each filtered trajectory variant, the preset negative value, the preset scoring threshold, or the intra-group reward score is used as the global reward score.

[0009] In one embodiment of this application, the process of filtering all selected trajectory variants and the smooth exploration trajectory based on the global reward score of each selected trajectory variant and the global reward score of the smooth exploration trajectory to obtain the optimal driving trajectory includes: merging all selected trajectory variants and the smooth exploration trajectory to form a candidate trajectory set; if there is a trajectory in the candidate trajectory set with a global reward score of a preset negative value, then removing the trajectory with the global reward score of the preset negative value from the candidate trajectory set to obtain an optimized trajectory set; if there is no trajectory with a global reward score of the preset negative value in the candidate trajectory set, then using the candidate trajectory set as the optimized trajectory set; and using the trajectory with the highest global reward score in the optimized trajectory set as the optimal driving trajectory.

[0010] In one embodiment of this application, the process of pre-constructing the truncated diffusion model of the hierarchical GRPO includes: acquiring first historical environmental features around the autonomous vehicle and first historical trajectory information of the autonomous vehicle; inputting the first historical environmental features into a pre-trained anchor point diffusion model to obtain a multi-anchor point predicted trajectory; adjusting the parameters of the pre-trained anchor point diffusion model to obtain the basic trajectory generator; superimposing the multi-anchor point predicted trajectory with a second multiplicative noise to obtain a sample exploration trajectory; and smoothing the sample exploration trajectory to obtain a smoothed sample trajectory; sampling the smoothed sample trajectory according to the first preset anchor point to obtain multiple sample trajectory variant groups; calculating the intra-group reward score of each sample trajectory variant within each sample trajectory variant group; selecting the sample trajectory variant with the maximum intra-group reward score within each sample trajectory variant group as the selected sample trajectory variant; and based on the preset anchor points... A truncated GRPO model is used to perform a global security assessment on each selected sample trajectory variant, resulting in a global security assessment result for each variant. For each selected sample trajectory variant, the intra-group reward score is corrected based on the global security assessment result, resulting in a global reward score for each variant. Furthermore, based on the pre-defined anchor-point truncated GRPO model, a global security assessment is performed on the smoothed sample trajectory, resulting in a global reward score for the smoothed sample trajectory. The pre-defined anchor-point truncated GRPO model is then adjusted to obtain the anchor-point truncated GRPO model. Based on the global reward score of the smoothed sample trajectory, the global reward score of each selected sample trajectory variant, and the first historical trajectory information, a loss function is constructed. With the goal of minimizing the loss function, the parameters in the pre-defined hierarchical GRPO truncated diffusion model are iteratively updated to obtain the hierarchical GRPO truncated diffusion model.

[0011] In one embodiment of this application, the expression for the loss function includes: ,

[0012] in, Represents the loss function. This represents the reinforcement learning loss function. Indicates the weighting coefficient. The expression for the reinforcement learning loss function includes: ,in, This represents the reinforcement learning loss function. This represents the expectation, used to calculate the mean of the product of the global reward score and the probability for all trajectories. Representing the trajectory Global reward score, Indicates the current policy model Generate trajectory The probability; the expression of the imitation learning loss function includes: ,in, This represents the imitation learning loss function. Indicates the total number of trajectories. Denotes KL divergence, Indicating the trajectory in the reference strategy The probability distribution is obtained by cloning and training the reference strategy using the first historical trajectory information. Indicates the current policy model Trajectory The predicted probability distribution.

[0013] In one embodiment of this application, before inputting the first historical environment feature into the pre-trained anchor diffusion model, the trajectory generation method further includes: acquiring a second historical environment feature and second historical trajectory information; the second historical environment feature occurs before the first historical environment feature, and the second historical trajectory information occurs before the first historical trajectory information; superimposing Gaussian noise generated in a random manner onto the historical trajectory information to obtain a noisy trajectory sample; inputting the noisy trajectory sample, the second historical environment feature, and the second preset anchor point into the anchor diffusion model to obtain predicted noise; adjusting the parameters in the anchor diffusion model with the goal of minimizing the difference between the predicted noise and the Gaussian noise to obtain the pre-trained anchor diffusion model.

[0014] In one embodiment of this application, the process of obtaining the current environment features by encoding and fusing the current environment information includes: converting the image information to the coordinate system of the autonomous vehicle using camera extrinsic parameters to obtain image information in the vehicle coordinate system; converting the map information to the coordinate system of the autonomous vehicle using the positioning information of the autonomous vehicle to obtain map information in the vehicle coordinate system; encoding the image information in the vehicle coordinate system to obtain image bird's-eye view features; encoding the map information in the vehicle coordinate system to obtain map bird's-eye view features; and fusing the image bird's-eye view features and the map bird's-eye view features at multiple scales to obtain the current environment features.

[0015] In one embodiment of this application, the calculation formula for the multi-anchor-point exploration trajectory includes: ,in, This indicates the multi-anchor point exploration trajectory in the th... The state of the time step, Indicates the candidate trajectory of multiple anchor points in the th... The state of the time step, Indicates the first The first multiplicative noise at the time step; the second The formula for calculating the first multiplicative noise at the time step includes: ,in, Indicates the first The first multiplicative noise at the time step, Indicates the first Longitudinal noise at time step, Indicates the first Lateral noise at the time step.

[0016] According to another aspect of the embodiments of this application, a trajectory generation system based on a hierarchical GRPO truncated diffusion model is also provided, comprising: a feature acquisition module for acquiring current environmental features around an autonomous vehicle; the current environmental features are obtained by encoding and fusing current environmental information; the current environmental information includes: image information and map information; a trajectory generation module for generating multi-anchor candidate trajectories using the current environmental features as anchor point trajectory generation conditions through a basic trajectory generator; a trajectory smoothing module for superimposing the multi-anchor candidate trajectories with a first multiplicative noise to obtain a multi-anchor exploration trajectory; and performing smoothing filtering on the multi-anchor exploration trajectory to obtain a smooth exploration trajectory; and an intra-group evaluation module for sampling the smooth exploration trajectory according to a first preset anchor point to obtain multiple trajectory variant groups; and calculating each trajectory variant group. The system includes: a group-based reward score for each trajectory variant; selecting the trajectory variant with the highest group-based reward score within each trajectory variant group as the selected trajectory variant; a global evaluation module, used to perform a global safety evaluation on each selected trajectory variant based on the anchor-truncation GRPO model, obtaining a global safety evaluation result for each selected trajectory variant; for each selected trajectory variant, correcting the group-based reward score based on the global safety evaluation result, obtaining a global reward score for each selected trajectory variant; and performing a global safety evaluation on the smooth exploration trajectory based on the anchor-truncation GRPO model, obtaining a global reward score for the smooth exploration trajectory; and a trajectory selection module, used to select all selected trajectory variants and the smooth exploration trajectory based on the global reward score of each selected trajectory variant and the global reward score of the smooth exploration trajectory, to obtain the optimal driving trajectory.

[0017] The beneficial effects of this application are as follows: This application obtains the current environmental features around the autonomous vehicle, uses these features as anchor point trajectory generation conditions, generates multi-anchor point candidate trajectories through a basic trajectory generator, superimposes the multi-anchor point candidate trajectories with a first multiplicative noise to obtain a multi-anchor point exploration trajectory, and performs smoothing filtering on the multi-anchor point exploration trajectory to obtain a smooth exploration trajectory. According to a first preset anchor point, the smooth exploration trajectory is sampled to obtain multiple trajectory variant groups. The intra-group reward score of each trajectory variant within each trajectory variant group is calculated. The trajectory variant with the highest intra-group reward score within each trajectory variant group is selected as the filtered trajectory variant. Based on the inter-anchor truncated GRPO model, a global safety assessment is performed on each filtered trajectory variant to obtain the global safety assessment result for each filtered trajectory variant. For each filtered trajectory variant, the intra-group reward score is corrected based on the global safety assessment result to obtain the global safety assessment result for each filtered trajectory variant. The system employs a hierarchical GRPO truncated diffusion model to perform a global safety assessment of smooth exploration trajectories, resulting in a global reward score for each trajectory variant. Based on the global reward scores of each selected trajectory variant and the smooth exploration trajectory, all selected trajectory variants and smooth exploration trajectories are filtered to obtain the optimal driving trajectory. This process, through the hierarchical GRPO truncated diffusion model, alleviates the contradiction between quality and diversity caused by incomplete supervision in the anchor-point diffusion model. It effectively constrains low-quality patterns and explores better strategies while maintaining trajectory diversity. By assigning intra-group reward scores to each trajectory variant and selecting the trajectory variant with the highest intra-group reward score within each group as the selected trajectory variant, the optimization process ensures that it respects and preserves the fundamentally different driving intentions defined by the preset anchor points, avoiding the pitfalls of traditional RL (Reinforcement Learning). Learning methods may lead to pattern collapse; the design of the anchor-point truncated GRPO model introduces a simple and effective safety constraint mechanism, which punishes any collision behavior, thereby greatly improving the overall safety of the hierarchical GRPO-based truncated diffusion model trajectory generation system without sacrificing diversity; the use of scale-adaptive multiplicative noise enables the trajectory exploration process to generate smooth trajectories that conform to vehicle dynamics, improving exploration efficiency and reducing invalid exploration; the hierarchical GRPO-based truncated diffusion model can be used as a later fine-tuning stage in the anchor-point-based diffusion model planner.

[0018] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0020] Figure 1 This is a schematic diagram illustrating an exemplary system architecture as shown in an exemplary embodiment of this application;

[0021] Figure 2 This is a flowchart illustrating an exemplary embodiment of the present application of a method for generating truncation diffusion model trajectories based on hierarchical GRPO;

[0022] Figure 3 This is a flowchart illustrating a pre-constructed hierarchical GRPO truncated diffusion model trajectory, as shown in an exemplary embodiment of this application.

[0023] Figure 4 This is a flowchart illustrating a method for generating truncation diffusion model trajectories based on hierarchical GRPO, as shown in another exemplary embodiment of this application.

[0024] Figure 5 This is a flowchart illustrating an exemplary embodiment of the present application of generating an optimized trajectory set using a truncated diffusion model based on a hierarchical GRPO;

[0025] Figure 6 This is a schematic diagram illustrating the effect of not performing advantage truncation optimization in an exemplary embodiment of this application;

[0026] Figure 7 This is a schematic diagram illustrating the effect after advantage truncation optimization, as shown in an exemplary embodiment of this application;

[0027] Figure 8 This is a block diagram illustrating a truncated diffusion model trajectory generation system based on hierarchical GRPO, as shown in an exemplary embodiment of this application. Detailed Implementation

[0028] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0029] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. The drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0030] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the present application. However, it will be apparent to those skilled in the art that embodiments of the present application may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the present application.

[0031] The implementation details of the technical solutions in the embodiments of this application are described in detail below:

[0032] Figure 1 This is a schematic diagram illustrating an exemplary system architecture as shown in an exemplary embodiment of this application, such as... Figure 1 , refer to Figure 1 As shown, the system architecture may include a data acquisition device 101 and an electronic device 102. The electronic device 102 may be at least one of a desktop graphics processing unit (GPU) computer, a GPU computing cluster, or a neural network computer. Those skilled in the art can use the electronic device 102 to acquire the current environmental features surrounding the autonomous vehicle, using these features as anchor point trajectory generation conditions. A multi-anchor point candidate trajectory is generated through a basic trajectory generator. This candidate trajectory is then superimposed with a first multiplicative noise to obtain a multi-anchor point exploration trajectory. The multi-anchor point exploration trajectory is then smoothed to obtain a smooth exploration trajectory. Following a first preset anchor point, the smooth exploration trajectory is sampled to obtain multiple trajectory variant groups. The intra-group reward score for each trajectory variant within each trajectory variant group is calculated. The trajectory variant with the highest intra-group reward score within each trajectory variant group is selected as the screening trajectory. The system employs a variant model based on the truncated GRPO model between anchor points. A global safety assessment is performed on each selected trajectory variant to obtain a global safety assessment result. For each selected trajectory variant, the intra-group reward score is corrected based on the global safety assessment result to obtain a global reward score. Furthermore, based on the truncated GRPO model between anchor points, a global safety assessment is performed on the smooth exploration trajectory to obtain a global reward score. Based on the global reward scores of each selected trajectory variant and the smooth exploration trajectory, all selected trajectory variants and smooth exploration trajectories are filtered to obtain the optimal driving trajectory. The data acquisition device 101 collects the current environmental features around the autonomous vehicle and provides them to the electronic device 102 for processing.

[0033] Schematic, after acquiring the current environmental features around the autonomous vehicle in the acquisition device 101, the electronic device 102 uses these current environmental features as anchor point trajectory generation conditions. It generates multi-anchor point candidate trajectories through a basic trajectory generator, superimposes these candidate trajectories with a first multiplicative noise to obtain a multi-anchor point exploration trajectory, and then smooths and filters the multi-anchor point exploration trajectory to obtain a smooth exploration trajectory. According to a first preset anchor point, the smooth exploration trajectory is sampled to obtain multiple trajectory variant groups. The intra-group reward score for each trajectory variant within each trajectory variant group is calculated, and the trajectory variant with the highest intra-group reward score is selected. The trajectory variants with reward scores are used as selection trajectory variants. Based on the truncated GRPO model between anchor points, a global safety assessment is performed on each selection trajectory variant to obtain a global safety assessment result for each selection trajectory variant. For each selection trajectory variant, the reward score within the group is corrected based on the global safety assessment result to obtain a global reward score for each selection trajectory variant. Furthermore, based on the truncated GRPO model between anchor points, a global safety assessment is performed on the smooth exploration trajectory to obtain a global reward score for the smooth exploration trajectory. Based on the global reward scores of each selection trajectory variant and the global reward scores of the smooth exploration trajectory, all selection trajectories are... The optimal driving trajectory is obtained by filtering trajectory variants and smooth exploration trajectories. This process, through a hierarchical GRPO truncated diffusion model, alleviates the contradiction between quality and diversity caused by incomplete supervision in the anchor-point diffusion model. It effectively constrains low-quality patterns and explores better strategies while maintaining trajectory diversity. By assigning intra-group reward scores to each trajectory variant and selecting the variant with the highest intra-group reward score within each group, the optimization process respects and preserves the fundamentally different driving intentions defined by the preset anchor points, avoiding pattern collapse that may occur with traditional RL methods. The design of the inter-anchor-point truncated GRPO model introduces a simple yet effective safety constraint mechanism, penalizing any collision behavior. This significantly improves the overall safety of the hierarchical GRPO-based truncated diffusion model trajectory generation system without sacrificing diversity. The use of scale-adaptive multiplicative noise enables the trajectory exploration process to generate smooth trajectories that conform to vehicle dynamics, improving exploration efficiency and reducing ineffective exploration. The hierarchical GRPO-based truncated diffusion model can be used as a later fine-tuning stage in the anchor-point-based diffusion model planner.

[0034] It should be noted that the method for generating truncated diffusion model trajectories based on hierarchical GRPO provided in this application embodiment is generally executed by electronic device 102, and correspondingly, the system for generating truncated diffusion model trajectories based on hierarchical GRPO is generally set in electronic device 102.

[0035] The implementation details of the technical solutions in the embodiments of this application are described in detail below:

[0036] Figure 2 This is a flowchart illustrating an exemplary embodiment of the truncated diffusion model trajectory generation method based on hierarchical GRPO, as shown in this application. (Refer to...) Figure 2 As shown, the method for generating truncation diffusion model trajectories based on hierarchical GRPO includes at least steps S210 to S260, which are described in detail below:

[0037] In step S210, the current environmental features surrounding the autonomous vehicle are acquired. In one embodiment of this application, the current environmental features are obtained by encoding and fusing current environmental information; the current environmental information includes image information and map information. The image information is acquired by an onboard camera, and the map information is determined based on the location information of the autonomous vehicle and a preset range. In step S220, using the current environmental features as anchor point trajectory generation conditions, a multi-anchor point candidate trajectory is generated by a basic trajectory generator. In one embodiment of this application, the basic trajectory generator is a pre-trained anchor point diffusion model obtained by parameter adjustment. For each anchor point, the pre-trained anchor point diffusion model generates a candidate trajectory from the distribution guided by that anchor point through a truncated diffusion process.

[0038] In step S230, the multi-anchor candidate trajectory is superimposed with the first multiplicative noise to obtain the multi-anchor exploration trajectory; and the multi-anchor exploration trajectory is smoothed by filtering to obtain the smoothed exploration trajectory. In one embodiment of this application, the smoothing filtering method can be moving average filtering, Gaussian filtering, Kalman filtering, etc. The calculation formula for the multi-anchor exploration trajectory includes:

[0039] Equation (1)

[0040] in, This indicates the multi-anchor point exploration trajectory in the th... The state of the time step, Indicates the candidate trajectory of multiple anchor points in the th... The state of the time step, Indicates the first The first multiplicative noise at the time step;

[0041] In the The formula for calculating the first multiplicative noise at the time step includes:

[0042] Equation (2)

[0043] in, Indicates the first The first multiplicative noise at the time step, Indicates the first The longitudinal noise of the time step is determined by factors such as the sampling noise in the normal distribution and the velocity at the current moment. Indicates the first Lateral noise at a time step is determined by factors such as the distance the autonomous vehicle deviates from the lane centerline, road curvature, and lane-changing intentions.

[0044] In one embodiment of this application, by introducing a first multiplicative noise into the multi-anchor candidate trajectory, a smooth and dynamically feasible exploration trajectory can be generated, avoiding the jagged and discontinuous trajectory caused by additive noise.

[0045] In step S240, the smooth exploration trajectory is sampled according to the first preset anchor point to obtain multiple trajectory variant groups; the intra-group reward score of each trajectory variant within each trajectory variant group is calculated; and the trajectory variant with the maximum intra-group reward score within each trajectory variant group is selected as the filtered trajectory variant. In one embodiment of this application, the first preset anchor point is set according to the actual situation, such as following, left turn, right turn, straight ahead, etc. The process of sampling the smooth exploration trajectory according to the first preset anchor point to obtain multiple trajectory variant groups; calculating the intra-group reward score of each trajectory variant within each trajectory variant group; and selecting the trajectory variant with the maximum intra-group reward score within each trajectory variant group as the filtered trajectory variant is only performed within the same driving intention (i.e., within the group), avoiding undue competition between different driving intentions (such as left turn and straight ahead), thereby protecting the multimodal characteristics of the model. Based on the first preset anchor point, the smooth exploration trajectory is sampled to obtain multiple trajectory variant groups; the intra-group reward score of each trajectory variant in each trajectory variant group is calculated; the process of selecting the trajectory variant with the maximum intra-group reward score in each trajectory variant group can also be implemented through the GRPO module in the anchor point.

[0046] In step S250, a global security assessment is performed on each selected trajectory variant based on the anchor-truncation GRPO model to obtain a global security assessment result for each selected trajectory variant. For each selected trajectory variant, the intra-group reward score is corrected based on the global security assessment result to obtain a global reward score for each selected trajectory variant. Furthermore, a global security assessment is performed on the smooth exploration trajectory based on the anchor-truncation GRPO model to obtain a global reward score for the smooth exploration trajectory. In one embodiment of this application, the global security assessment result includes: collision presence and non-collision absence. After obtaining the selected trajectory variants within each trajectory variant group, a cross-anchor-point global perspective is introduced to address the information silo problem that may arise from intra-group comparisons. Specifically, a unified global security assessment is performed on each selected trajectory variant and the smooth exploration trajectory. Based on the global estimation result, trajectories that collide are strongly penalized regardless of their intra-group relative advantage. Trajectories that collide retain their intra-group positive advantage, while their negative advantage is truncated to zero. This provides a clear and consistent learning signal: encouraging improvement under safe conditions.

[0047] In one embodiment of this application, the process of performing a global safety assessment on a smooth exploration trajectory based on the anchor-point truncated GRPO model to obtain a global reward score for the smooth exploration trajectory includes: calculating a preset reward score for the corresponding smooth exploration trajectory based on the efficiency reward, comfort reward, and rule compliance reward for each smooth exploration trajectory; performing a global safety assessment on the smooth exploration trajectory based on the anchor-point truncated GRPO model to obtain a global safety assessment result for the smooth exploration trajectory; and correcting the preset reward score of the smooth exploration trajectory according to the global safety assessment result to obtain a global reward score for the smooth exploration trajectory. The process of calculating the preset reward score for each smooth exploration trajectory is the same as the process of calculating the intra-group reward score for each trajectory variant within each trajectory variant group. The process of correcting the preset reward score of the smooth exploration trajectory according to the global safety assessment result to obtain a global reward score for the smooth exploration trajectory is the same as the process of correcting the intra-group reward score for each selected trajectory variant based on the global safety assessment result to obtain a global reward score for each selected trajectory variant.

[0048] In step S260, all the selected trajectory variants and smooth exploration trajectories are filtered based on the global reward score of each selected trajectory variant and the global reward score of the smooth exploration trajectory to obtain the optimal driving trajectory. In one embodiment of this application, a hierarchical GRPO truncated diffusion model alleviates the contradiction between quality and diversity caused by incomplete supervision in imitation learning in anchor point diffusion models. This model effectively constrains low-quality patterns and explores better strategies while maintaining trajectory diversity. By assigning intra-group reward scores to each trajectory variant and selecting the variant with the highest intra-group reward score within each group, the optimization process respects and preserves the fundamentally different driving intentions defined by preset anchor points, avoiding pattern collapse that may occur with traditional RL methods. The design of the inter-anchor point truncated GRPO model introduces a simple yet effective safety constraint mechanism, penalizing any collision behavior. This significantly improves the overall safety of the hierarchical GRPO-based truncated diffusion model trajectory generation system without sacrificing diversity. The use of scale-adaptive multiplicative noise enables the trajectory exploration process to generate smooth trajectories that conform to vehicle dynamics, improving exploration efficiency and reducing ineffective exploration. The hierarchical GRPO-based truncated diffusion model can be used as a later fine-tuning stage in an anchor point-based diffusion model planner.

[0049] In one embodiment of this application, the process of calculating the in-group reward score for each trajectory variant within each trajectory variant group includes:

[0050] The efficiency reward, comfort reward, and rule compliance reward are obtained for each trajectory variant within each trajectory variant group. In one embodiment of this application, the formula for calculating the efficiency reward is as follows:

[0051] Equation (3)

[0052] in, Indicates an efficiency reward. Indicates the speed reward weight. Indicates the first The longitudinal speed at the time step. Indicates the target speed. Indicates speed tolerance. Indicates the progress reward weight. Indicates the displacement increment. This represents the stagnation penalty coefficient (e.g., 5.0). The sum of the speed reward weight and the progress reward weight equals 1.

[0053] The formula for calculating comfort reward is as follows:

[0054] Equation (4)

[0055] in, Indicates a comfort reward. Indicates the longitudinal acceleration reward weight. Indicates the first longitudinal acceleration at time step Indicates the first Lateral acceleration at time step Indicates the weight of the lateral acceleration reward. Indicates the first accelerometer at time step This indicates the weight of the acceleration reward. This represents the maximum value of longitudinal acceleration. This represents the maximum value of lateral acceleration. This represents the maximum value of the jerk. The sum of the longitudinal acceleration bonus weight, the lateral acceleration bonus weight, and the jerk bonus weight equals 1.

[0056] The formula for calculating the rule compliance reward is as follows:

[0057] Equation (5)

[0058] in, This indicates that following the rules will be rewarded. Indicates lane departure weight. This represents the absolute value of the lateral deviation distance, used to characterize the lateral distance between the autonomous vehicle and the lane centerline. This represents half the lane width. This indicates the weight of the penalty for running a red light. This indicates an indicator function; it is 1 when a violation occurs, and 0 otherwise. This represents the hard penalty coefficient. The sum of the lane departure penalty weight and the red light violation penalty weight equals 1.

[0059] Grouped by trajectory variant, the efficiency reward, comfort reward, and rule compliance reward for each trajectory variant are linearly weighted and summed to obtain the comprehensive reward score for each trajectory variant within each trajectory variant group. In one embodiment of this application, the formula for calculating the comprehensive reward score is as follows:

[0060] Equation (6)

[0061] in, This indicates the overall reward score. Indicates the efficiency reward weight. Indicates an efficiency reward. Indicates the comfort reward weight. Indicates a comfort reward. This indicates that the rules follow reward weighting. This indicates the reward for rule compliance. The sum of the efficiency reward weight, comfort reward weight, and rule compliance reward weight equals 1.

[0062] Based on the comprehensive reward score of each trajectory variant within each trajectory variant group, the mean and standard deviation of the comprehensive reward score within each trajectory variant group are calculated. Then, based on the mean and standard deviation of the comprehensive reward score within each trajectory variant group, the comprehensive reward score of each trajectory variant within the corresponding group is normalized to obtain the intra-group reward score of each trajectory variant within each trajectory variant group. In one embodiment of this application, taking a certain trajectory variant group as an example, the calculation formula for the intra-group reward score of the trajectory variants within this trajectory variant group is as follows:

[0063] Equation (7)

[0064] in, Indicates the first [item] within the trajectory variant group. Intra-group reward scores for each trajectory variant Indicates the first [item] within the trajectory variant group. The overall reward score for each trajectory variant. This represents the mean of the overall reward score within the group of this trajectory variant. This represents the standard deviation of the overall reward score.

[0065] In one embodiment of this application, the operation is performed only within the same driving intent (i.e., within a group) to avoid undue competition between different driving intents (such as left turn and straight drive), thereby protecting the multimodal characteristics of the model.

[0066] In one embodiment of this application, the process of correcting the intra-group reward score based on the global security assessment result for each selected trajectory variant to obtain the global reward score for each selected trajectory variant includes:

[0067] For each selected trajectory variant, if the global safety assessment result indicates a collision, the group reward score is corrected to a preset negative value; if the global safety assessment result indicates no collision and the group reward score is less than a preset score threshold, the group reward score is corrected to the preset score threshold; if the global safety assessment result indicates no collision and the group reward score is greater than or equal to the preset score threshold, no correction is made to the group reward score. In one embodiment of this application, the preset negative value can be set to -1 or other values, and the preset score threshold can be set to 0 or other values.

[0068] For each selected trajectory variant, a preset negative value, a preset scoring threshold, or an intra-group reward score is used as the global reward score. In one embodiment of this application, the formula for calculating the global reward score is as follows:

[0069] Equation (8)

[0070] in, Indicates the first Global reward score for each trajectory variant Indicates the first digit within a certain trajectory variant group Intra-group bonus score for each trajectory variant.

[0071] In one embodiment of this application, after obtaining the selected trajectory variants within each trajectory variant group, a global perspective across anchor points is introduced to address the information silo problem that may result from intra-group comparisons. Specifically, a unified global safety assessment is performed through each selected trajectory variant and smoothed exploration trajectory. Based on the global estimation results, trajectories that collide are strongly penalized regardless of their relative intra-group advantage. Trajectories that collide retain their positive intra-group advantage, while their negative advantage is truncated to zero. This provides a clear and consistent learning signal that encourages improvement under safe conditions.

[0072] In one embodiment of this application, the process of filtering all selected trajectory variants and smooth exploration trajectories based on the global reward score of each selected trajectory variant and the global reward score of the smooth exploration trajectory to obtain the optimal driving trajectory includes:

[0073] All selected trajectory variants and smoothed exploration trajectories are merged to form a candidate trajectory set. In one embodiment of this application, the candidate trajectory set includes smoothed exploration trajectories and trajectory variants with the highest in-group reward score within each trajectory variant group as selected trajectory variants. This achieves screening of trajectory variants within each trajectory variant group, ensuring not only the in-group advantage of the trajectory but also avoiding the limitations of local optima. Furthermore, preliminary screening helps reduce unnecessary computational overhead, thereby significantly improving the algorithm's operating efficiency while maintaining trajectory quality.

[0074] If a trajectory in the candidate trajectory set has a global reward score that is negative by a preset value, then that trajectory is removed from the candidate trajectory set to obtain an optimized trajectory set. In one embodiment of this application, a trajectory with a global reward score that is negative by a preset value indicates that the trajectory has collided. Therefore, it is necessary to remove the colliding trajectories from the candidate trajectory set to further improve the trajectory quality while ensuring the diversity of the trajectories.

[0075] If there is no trajectory in the candidate trajectory set with a preset negative global reward score, then the candidate trajectory set is used as the optimized trajectory set. In one embodiment of this application, a trajectory with a preset negative global reward score indicates that a trajectory collision has occurred. If there is no collision-caused trajectory in the candidate trajectory set, then there is no need to change the candidate trajectory set.

[0076] The trajectory with the highest global reward score in the optimized trajectory set is selected as the optimal driving trajectory. In one embodiment of this application, the purpose of selecting the trajectory with the highest global reward score in the optimized trajectory set as the optimal driving trajectory is to select the trajectory with the best evaluation indicators in terms of efficiency, comfort, and rule compliance from the trajectories that do not cause collisions, so as to improve the safety and comfort of driving autonomous vehicles.

[0077] In one embodiment of this application, the process of pre-constructing a truncated diffusion model of a hierarchical GRPO includes:

[0078] The system acquires first historical environmental features surrounding the autonomous vehicle and first historical trajectory information of the autonomous vehicle. In one embodiment of this application, the first historical environmental features are obtained by encoding and fusing first historical environmental information, which includes first historical image information and first historical map information, etc. The first historical trajectory information includes: position information at different historical times, historical velocity information at different historical times, historical acceleration information at different historical times, historical orientation angle information at different historical times, and motion information at different historical times, etc.

[0079] The first historical environmental features are input into a pre-trained anchor point diffusion model to obtain a multi-anchor point predicted trajectory. In one embodiment of this application, the pre-trained anchor point diffusion model is adjusted to obtain a basic trajectory generator. For each anchor point, the pre-trained anchor point diffusion model generates a candidate trajectory from the distribution guided by that anchor point through a truncated diffusion process.

[0080] The multi-anchor-point predicted trajectory is superimposed with the second multiplicative noise to obtain the sample exploration trajectory; then, the sample exploration trajectory is smoothed by filtering to obtain a smoothed sample trajectory. In one embodiment of this application, the intensity of the second multiplicative noise is different from that of the first multiplicative noise. The calculation formula for the superimposed sample exploration trajectory is the same as that for the multi-anchor-point exploration trajectory. The smoothed sample trajectory can be obtained by using moving average filtering, Gaussian filtering, Kalman filtering, etc.

[0081] Smooth sample trajectories are sampled according to a first preset anchor point to obtain multiple sample trajectory variant groups; the intra-group reward score of each sample trajectory variant within each sample trajectory variant group is calculated; and the sample trajectory variant with the highest intra-group reward score within each sample trajectory variant group is selected as the filtered sample trajectory variant. In one embodiment of this application, the first preset anchor point is set according to actual conditions, such as following a vehicle, turning left, turning right, or going straight. The process of sampling smooth sample trajectories according to the first preset anchor point to obtain multiple sample trajectory variant groups; calculating the intra-group reward score of each sample trajectory variant within each sample trajectory variant group; and selecting the sample trajectory variant with the highest intra-group reward score within each sample trajectory variant group as the filtered sample trajectory variant is implemented through the Intra-Anchor GRPO (Intra-Anchor GRPO) module before parameter adjustment.

[0082] Based on a pre-defined anchor-truncation GRPO model, a global security assessment is performed on each selected sample trajectory variant to obtain a global security assessment result for each selected sample trajectory variant. For each selected sample trajectory variant, the intra-group reward score is corrected based on the global security assessment result to obtain a global reward score for each selected sample trajectory variant. Furthermore, based on the pre-defined anchor-truncation GRPO model, a global security assessment is performed on smoothed sample trajectories to obtain a global reward score for smoothed sample trajectories. In one embodiment of this application, the pre-defined anchor-truncation GRPO model is obtained after parameter adjustment; the process of correcting the intra-group reward score for each selected sample trajectory variant based on the global security assessment result to obtain a global reward score for each selected sample trajectory variant is the same as the process of correcting the intra-group reward score for each selected trajectory variant based on the global security assessment result to obtain a global reward score for each selected trajectory variant.

[0083] Based on the global reward score of the smoothed sample trajectory, the global reward score of each selected sample trajectory variant, and the first historical trajectory information, a loss function is constructed. With the goal of minimizing the loss function, the parameters in the pre-defined hierarchical GRPO truncated diffusion model are iteratively updated to obtain the hierarchical GRPO truncated diffusion model. In one embodiment of this application, the expression of the loss function includes:

[0084] Equation (9)

[0085] in, Represents the loss function. This represents the reinforcement learning loss function. Indicates the weighting coefficient. This represents the loss function for imitation learning;

[0086] The expression for the reinforcement learning loss function includes:

[0087] Equation (10)

[0088] in, This represents the reinforcement learning loss function. This represents the expectation, used to calculate the mean of the product of the global reward score and the probability for all trajectories. Representing the trajectory Global reward score, Indicates the current policy model Generate trajectory The probability of;

[0089] The expression for the imitation learning loss function includes:

[0090] Equation (11)

[0091] in, This represents the imitation learning loss function. Indicates the total number of trajectories. Denotes KL divergence, Indicating the trajectory in the reference strategy The probability distribution is obtained by cloning and training the reference strategy using the first historical trajectory information. Indicates the current policy model Trajectory The predicted probability distribution.

[0092] In one embodiment of this application, a joint loss function is constructed using reinforcement learning and imitation learning loss functions. This helps prevent the truncated diffusion model of the pre-defined hierarchical GRPO from forgetting its basic driving abilities during training. Simultaneously, imitation learning preserves effective information from historical trajectories, thereby improving the performance of the truncated diffusion model while ensuring the stability of its basic driving abilities. This joint training method enables the truncated diffusion model of the pre-defined hierarchical GRPO to adapt to new environments while maintaining its grasp of historical knowledge in complex and ever-changing driving scenarios, avoiding behavioral deviations caused by over-optimization. Furthermore, by adjusting the weight coefficients in the loss function, the balance between reinforcement learning and imitation learning can be flexibly controlled, further enhancing the generalization ability and robustness of the truncated diffusion model of the pre-defined hierarchical GRPO.

[0093] During training, the pre-defined hierarchical GRPO truncated diffusion model iteratively approaches the optimal parameter configuration. In each iteration, the model compares the trajectory generated by the current policy with historical trajectories in the reference policy and optimizes the policy by combining the global reward score. This process not only effectively reduces the search range of the model in the high-dimensional trajectory space during training but also significantly improves training efficiency. By grouping and filtering multi-anchor predicted trajectories, the model can quickly locate high-quality candidate trajectories while ensuring trajectory diversity, laying the foundation for subsequent global evaluation.

[0094] In one embodiment of this application, the hierarchical GRPO truncated diffusion model includes: an input layer, a basic trajectory generator, an exploration noise module, an intra-anchor GRPO module, an inter-anchor truncated GRPO module, and an output layer. The input layer acquires the current environmental features surrounding the autonomous vehicle; the basic trajectory generator uses the current environmental features as anchor trajectory generation conditions to generate multi-anchor candidate trajectories; the exploration noise module superimposes the multi-anchor candidate trajectories with a first multiplicative noise to obtain multi-anchor exploration trajectories; and performs smoothing filtering on the multi-anchor exploration trajectories to obtain smooth exploration trajectories; the intra-anchor GRPO module samples the smooth exploration trajectories according to a first preset anchor point to obtain multiple trajectory variant groups; calculates the intra-group reward score for each trajectory variant within each trajectory variant group; and assigns a score to each trajectory variant group containing... The trajectory variant with the highest intra-group reward score is used as the selected trajectory variant; the anchor-point truncated GRPO module is used to perform a global safety assessment on each selected trajectory variant to obtain the global safety assessment result for each selected trajectory variant; for each selected trajectory variant, the intra-group reward score is corrected based on the global safety assessment result to obtain the global reward score for each selected trajectory variant; and a global safety assessment is performed on the smooth exploration trajectory to obtain the global reward score for the smooth exploration trajectory; the output layer is used to filter all selected trajectory variants and smooth exploration trajectories based on the global reward score of each selected trajectory variant and the global reward score of the smooth exploration trajectory to obtain the optimal driving trajectory.

[0095] In one embodiment of this application, before inputting the first historical environmental features into the pre-trained anchor diffusion model, the trajectory generation method further includes:

[0096] The process involves acquiring second historical environmental features and second historical trajectory information. The second historical environmental features occur before the first historical environmental features, and the second historical trajectory information occurs before the first historical trajectory information. In one embodiment of this application, the second historical environmental features are obtained by encoding and fusing historical environmental information, which includes second historical image information and second historical map information, etc. The second historical trajectory information includes: position information at different historical moments, historical velocity information at different historical moments, historical acceleration information at different historical moments, historical orientation angle information at different historical moments, and motion information at different historical moments, etc.

[0097] Randomly generated Gaussian noise is superimposed onto historical trajectory information to obtain noisy trajectory samples. In one embodiment of this application, superimposing randomly generated Gaussian noise onto historical trajectory information helps increase the diversity of training data. This allows for the effective simulation of uncertainties in real-world driving scenarios through processing and analysis of the noisy trajectory samples, making the anchor point diffusion model more adaptable to complex environments. Furthermore, the noisy trajectory samples can help the anchor point diffusion model better learn potential patterns in historical trajectories, avoiding overfitting problems caused by overly simplistic data sets.

[0098] The noisy trajectory sample, second historical environmental features, and second preset anchor points are input into an anchor point diffusion model to obtain predicted noise. In one embodiment of this application, the anchor point diffusion model is used to gradually remove noise components from the noisy trajectory sample through a multi-step diffusion process, given the input noisy trajectory sample, second historical environmental features, and second preset anchor points, thereby generating a smoother predicted trajectory that conforms to actual driving behavior. During this process, the anchor point diffusion model can accurately model the noise distribution by combining information from the second historical environmental features and the second preset anchor points, and adjust the intensity and direction of the noise in each diffusion step to ensure that the generated predicted trajectory retains the core features of the historical trajectory while adapting to changes in the current environment. Furthermore, after obtaining the predicted noise, a clear trajectory sample corresponding to the noisy trajectory sample can be reconstructed by inverse processing the predicted noise. The second preset anchor point can be the same as or different from the first preset anchor point.

[0099] With the goal of minimizing the difference between predicted noise and Gaussian noise, the parameters in the anchor point diffusion model are adjusted to obtain a pre-trained anchor point diffusion model. In one embodiment of this application, the difference between predicted noise and Gaussian noise is characterized by a mean squared error loss function. By adjusting the parameters in the anchor point diffusion model, the accuracy of the pre-trained anchor point diffusion model in predicting noise distribution and denoising trajectory in complex scenarios is improved, thereby enabling the efficient generation of diverse and safety-consistent multi-anchor point candidate trajectories based on different driving intention anchor points.

[0100] In one embodiment of this application, the process of obtaining current environmental characteristics by encoding and fusing current environmental information includes:

[0101] Image information is transformed to the coordinate system of the autonomous vehicle using camera extrinsic parameters, resulting in image information in the vehicle coordinate system. Similarly, map information is transformed to the coordinate system of the autonomous vehicle using its positioning information, resulting in map information in the vehicle coordinate system. In one embodiment of this application, the process of transforming surrounding multi-view images to the coordinate system of the autonomous vehicle using camera extrinsic parameters to obtain multi-view images in the vehicle coordinate system can be implemented using conversion tools such as OpenCalib, CamOdoCal, and FAST-Calib. Similarly, the process of transforming map information to the coordinate system of the autonomous vehicle using its positioning information can be implemented using conversion tools such as PROJ and CoordTransform.

[0102] Image information in the vehicle coordinate system is image encoded to obtain bird's-eye view features; map information in the vehicle coordinate system is map encoded to obtain map bird's-eye view features. In one embodiment of this application, the process of image encoding of image information in the vehicle coordinate system to obtain bird's-eye view features can be implemented by an encoder, such as a BEVFormer encoder or a PETR (Position Embedding Transformation) encoder. The process of map encoding of map information in the vehicle coordinate system to obtain map bird's-eye view features can be implemented by a map encoder, such as a Unified Vector Encoder (UVE) or a Transformer-based map encoder.

[0103] The current environment features are obtained by multi-scale fusion of image bird's-eye view features and map bird's-eye view features. In one embodiment of this application, the multi-scale fusion of image bird's-eye view features and map bird's-eye view features can be implemented by referring to the process of multi-scale fusion of image bird's-eye view features and map bird's-eye view features in related technologies.

[0104] Figure 3 This is a flowchart illustrating a pre-constructed hierarchical GRPO truncated diffusion model trajectory, as shown in another exemplary embodiment of this application. Figure 3 As can be seen from the above, the process of pre-constructing a hierarchical GRPO truncated diffusion model includes: (1) The input layer obtains the first historical environmental features around the autonomous vehicle and the first historical trajectory information of the autonomous vehicle; (2) The generation layer includes: a pre-trained anchor point diffusion model and an exploration noise module. The pre-trained anchor point diffusion model is used to generate multi-anchor point predicted trajectories with the first historical environmental features as the anchor point trajectory generation condition; the exploration noise module is used to superimpose the multi-anchor point predicted trajectories with the second multiplicative noise to obtain sample exploration trajectories; and to perform smoothing filtering on the sample exploration trajectories to obtain smooth sample trajectories; (3) The reinforcement learning optimization layer includes: a preset anchor point GRPO module, a preset anchor point truncated GRPO model, a loss function establishment and model update module. The preset anchor point GRPO module is used to sample the smooth sample trajectories according to the first preset anchor point to obtain multiple sample trajectory variant groups; and to calculate the sample trajectory within each sample trajectory variant group. The process involves several steps: First, the within-group reward score of each trajectory variant is calculated. The trajectory variant with the highest within-group reward score within each group is selected as the filter trajectory variant. A pre-defined anchor-point truncated GRPO model is used to perform a global security assessment on each filter trajectory variant, yielding a global security assessment result. For each filter trajectory variant, the within-group reward score is adjusted based on the global security assessment result, resulting in a global reward score. A global security assessment is also performed on smoothed trajectories, yielding a global reward score for the smoothed trajectories. A loss function construction and model update module is used to construct a loss function based on the global reward score of the smoothed trajectories, the global reward score of each filter trajectory variant, and the first historical trajectory information. The parameters in the pre-defined hierarchical GRPO truncated diffusion model are iteratively updated with the goal of minimizing the loss function, resulting in the hierarchical GRPO truncated diffusion model. The pre-defined hierarchical GRPO truncated diffusion model includes an input layer, a generation layer, and a reinforcement learning optimization layer.

[0105] Figure 4 This is a flowchart illustrating a method for generating truncation diffusion models based on hierarchical GRPO, as shown in another exemplary embodiment of this application. Figure 4As shown, the process of generating the optimal driving trajectory based on the hierarchical GRPO truncated diffusion model includes: (1) the input layer obtains the current environmental features around the autonomous vehicle; (2) the generation layer includes: a basic trajectory generator and an exploration noise module. The basic trajectory generator is used to generate multi-anchor candidate trajectories with the current environmental features as the anchor point trajectory generation conditions; the exploration noise module is used to superimpose the multi-anchor candidate trajectories with the first multiplicative noise to obtain multi-anchor exploration trajectories; and to perform smoothing filtering on the multi-anchor exploration trajectories to obtain smooth exploration trajectories; (3) the reinforcement learning optimization layer includes: an intra-anchor GRPO module and an inter-anchor truncated GRPO model. The intra-anchor GRPO module is used to sample the smooth exploration trajectory according to the first preset anchor point to obtain multiple trajectory variant groups; and to calculate each trajectory... The intra-group reward score of each trajectory variant within the trajectory variant group; the trajectory variant with the largest intra-group reward score within each trajectory variant group is selected as the selected trajectory variant; the anchor-point truncated GRPO model is used to perform a global safety assessment on each selected trajectory variant to obtain the global safety assessment result of each selected trajectory variant; for each selected trajectory variant, the intra-group reward score is corrected by the global safety assessment result to obtain the global reward score of each selected trajectory variant; and is used to perform a global safety assessment on the smooth exploration trajectory to obtain the global reward score of the smooth exploration trajectory; (4) the output layer is used to filter all selected trajectory variants and smooth exploration trajectories based on the global reward score of each selected trajectory variant and the global reward score of the smooth exploration trajectory to obtain the optimal driving trajectory.

[0106] Figure 5 This is a flowchart illustrating an exemplary embodiment of the present application of generating an optimized trajectory set using a truncated diffusion model based on a hierarchical GRPO, as shown below. Figure 5 As shown, the process of generating an optimized trajectory set based on the hierarchical GRPO truncated diffusion model includes: (1) obtaining a first preset anchor point (e.g., following, left turn, right turn, straight ahead); (2) an anchor point definition layer is used to classify the intents of following, left turn, right turn, and straight ahead; (3) for each intent, an intra-group sampling layer samples G trajectory variants (e.g., trajectory variants) from the denoising process of the diffusion model. Trajectory variants Trajectory variants and trajectory variants (4) The GRPO module within each anchor point is used to calculate the intra-group reward score of each trajectory variant within each trajectory variant group; the trajectory variant with the largest intra-group reward score within each trajectory variant group is taken as the intra-group dominant trajectory variant; (5) The GRPO module between anchor points is used to perform a global safety assessment on each selected trajectory variant to obtain the global safety assessment result of each selected trajectory variant; for each selected trajectory variant, the intra-group reward score is corrected by the global safety assessment result to obtain the global reward score of each selected trajectory variant; all selected trajectory variants and smooth exploration trajectories are merged to form a candidate trajectory set, and the candidate trajectory set is taken as the optimized trajectory set.

[0107] Figure 6 This is a schematic diagram illustrating the effect of not performing advantage truncation optimization in an exemplary embodiment of this application, as shown below. Figure 6 As shown, the optimized trajectory set contains 5 trajectories. Blue represents right-turn trajectories, yellow represents left-turn trajectories, red represents trajectories where collisions occur, and green represents safe trajectories. The existence of trajectories where collisions occur greatly reduces the safety of autonomous driving.

[0108] Figure 7 This is a schematic diagram illustrating the effect of advantage truncation optimization in an exemplary embodiment of this application, as shown below. Figure 7 As shown, the optimized trajectory set contains four trajectories, where blue represents right-turn trajectories, yellow represents left-turn trajectories, and green represents safe trajectories. Due to the advantage truncation optimization performed in this application, trajectories that could lead to collisions are eliminated in advance, greatly improving the safety of autonomous driving.

[0109] The advantages of this application compared to existing technologies are shown in the table below:

[0110] Table 1

[0111]

[0112] As can be seen from Table 1, this application significantly improves the safety and comfort of the generated trajectory through hierarchical GRPO optimization; the GRPO mechanism within anchor points effectively protects the diversity of driving intentions and avoids pattern collapse; the safety constraint mechanism of truncating GRPO modules between anchor points greatly reduces the risk of collision; and the design of multiplicative noise and joint loss function makes RL training more stable and converges faster.

[0113] The following describes an embodiment of the apparatus described in this application, which can be used to execute the hierarchical GRPO-based truncated diffusion model trajectory generation system described in the above embodiments of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the hierarchical GRPO-based truncated diffusion model trajectory generation method described above in this application.

[0114] Figure 8 This is a block diagram illustrating a truncated diffusion model trajectory generation system based on hierarchical GRPO, as shown in an exemplary embodiment of this application.

[0115] like Figure 8 As shown, this exemplary truncated diffusion model trajectory generation system 800 based on hierarchical GRPO includes:

[0116] The feature acquisition module 801 is used to acquire the current environmental features around the autonomous vehicle.

[0117] The trajectory generation module 802 is used to generate multi-anchor-point candidate trajectories by using the current environmental features as anchor point trajectory generation conditions and through the basic trajectory generator.

[0118] The trajectory smoothing module 803 is used to superimpose the multi-anchor candidate trajectory with the first multiplicative noise to obtain the multi-anchor exploration trajectory; and to perform smoothing filtering on the multi-anchor exploration trajectory to obtain the smooth exploration trajectory.

[0119] The intra-group evaluation module 804 is used to sample the smooth exploration trajectory according to the first preset anchor point to obtain multiple trajectory variant groups; calculate the intra-group reward score of each trajectory variant in each trajectory variant group; and select the trajectory variant with the maximum intra-group reward score in each trajectory variant group as the selected trajectory variant.

[0120] The global assessment module 805 is used to perform a global safety assessment on each selected trajectory variant based on the anchor-truncation GRPO model, and obtain a global safety assessment result for each selected trajectory variant; for each selected trajectory variant, the reward score within the group is corrected based on the global safety assessment result, and a global reward score for each selected trajectory variant is obtained; and a global safety assessment is performed on the smooth exploration trajectory based on the anchor-truncation GRPO model, and a global reward score for the smooth exploration trajectory is obtained.

[0121] The trajectory filtering module 806 is used to filter all trajectory variants and smooth exploration trajectories based on the global reward score of each filtered trajectory variant and the global reward score of the smooth exploration trajectory to obtain the optimal driving trajectory.

[0122] In one embodiment of this application, the current environmental features are obtained by encoding and fusing the current environmental information; the current environmental information includes: image information and map information, the image information is collected by an on-board camera, and the map information is determined based on the location information of the autonomous vehicle and a preset range.

[0123] In one embodiment of this application, the basic trajectory generator is obtained by parameter tuning of a pre-trained anchor diffusion model. For each anchor, the pre-trained anchor diffusion model generates candidate trajectories from the distribution guided by that anchor through a truncated diffusion process.

[0124] In one embodiment of this application, the smoothing filter can be a moving average filter, a Gaussian filter, a Kalman filter, etc. The calculation formula for the multi-anchor point exploration trajectory is shown in equation (1). The formula for calculating the first multiplicative noise at the time step is shown in Equation (1).

[0125] In one embodiment of this application, by introducing a first multiplicative noise into the multi-anchor candidate trajectory, a smooth and dynamically feasible exploration trajectory can be generated, avoiding the jagged and discontinuous trajectory caused by additive noise.

[0126] In one embodiment of this application, the first preset anchor point is set according to the actual situation, such as following another vehicle, turning left, turning right, or going straight. Based on the first preset anchor point, the smooth exploration trajectory is sampled to obtain multiple trajectory variant groups; the intra-group reward score of each trajectory variant within each trajectory variant group is calculated; the process of selecting the trajectory variant with the highest intra-group reward score within each trajectory variant group is only performed within the same driving intention (i.e., within the group), avoiding undue competition between different driving intentions (such as turning left and going straight), thereby protecting the multimodal characteristics of the model. The process of sampling the smooth exploration trajectory based on the first preset anchor point to obtain multiple trajectory variant groups; calculating the intra-group reward score of each trajectory variant within each trajectory variant group; and selecting the trajectory variant with the highest intra-group reward score within each trajectory variant group can also be implemented through the GRPO module within the anchor point.

[0127] In one embodiment of this application, the global security assessment result includes: collision presence and non-collision absence. After obtaining the selected trajectory variants within each trajectory variant group, a global perspective across anchor points is introduced to address the information silo problem that may result from intra-group comparisons. Specifically, a unified global security assessment is performed through each selected trajectory variant and smoothed exploration trajectory. Based on the global estimation result, trajectories that collide are strongly penalized regardless of their relative intra-group advantage. Trajectories that collide retain their positive intra-group advantage, while their negative advantage is truncated to zero. This provides a clear and consistent learning signal: encouraging improvement under the premise of security.

[0128] In one embodiment of this application, the process of performing a global safety assessment on a smooth exploration trajectory based on the anchor-point truncated GRPO model to obtain a global reward score for the smooth exploration trajectory includes: calculating a preset reward score for the corresponding smooth exploration trajectory based on the efficiency reward, comfort reward, and rule compliance reward for each smooth exploration trajectory; performing a global safety assessment on the smooth exploration trajectory based on the anchor-point truncated GRPO model to obtain a global safety assessment result for the smooth exploration trajectory; and correcting the preset reward score of the smooth exploration trajectory according to the global safety assessment result to obtain a global reward score for the smooth exploration trajectory. The process of calculating the preset reward score for each smooth exploration trajectory is the same as the process of calculating the intra-group reward score for each trajectory variant within each trajectory variant group. The process of correcting the preset reward score of the smooth exploration trajectory according to the global safety assessment result to obtain a global reward score for the smooth exploration trajectory is the same as the process of correcting the intra-group reward score for each selected trajectory variant based on the global safety assessment result to obtain a global reward score for each selected trajectory variant.

[0129] In one embodiment of this application, a hierarchical GRPO truncated diffusion model alleviates the contradiction between quality and diversity caused by incomplete supervision in imitation learning in anchor point diffusion models. This model effectively constrains low-quality patterns and explores better strategies while maintaining trajectory diversity. By assigning intra-group reward scores to each trajectory variant and selecting the variant with the highest intra-group reward score within each group, the optimization process respects and preserves the fundamentally different driving intentions defined by preset anchor points, avoiding pattern collapse that may occur with traditional RL methods. The design of the inter-anchor point truncated GRPO model introduces a simple yet effective safety constraint mechanism, penalizing any collision behavior. This significantly improves the overall safety of the hierarchical GRPO-based truncated diffusion model trajectory generation system without sacrificing diversity. The use of scale-adaptive multiplicative noise enables the trajectory exploration process to generate smooth trajectories that conform to vehicle dynamics, improving exploration efficiency and reducing ineffective exploration. The hierarchical GRPO-based truncated diffusion model can be used as a later fine-tuning stage in an anchor point-based diffusion model planner.

[0130] It should be noted that the hierarchical GRPO-based truncated diffusion model trajectory generation system and the hierarchical GRPO-based truncated diffusion model trajectory generation method provided in the above embodiments belong to the same concept. The specific operation methods of each module and unit have been described in detail in the method embodiments and will not be repeated here. In practical applications, the hierarchical GRPO-based truncated diffusion model trajectory generation system provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0131] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A method for generating truncation diffusion model trajectories based on hierarchical GRPO, characterized in that, The trajectory generation method is based on a hierarchical GRPO truncated diffusion model, and the trajectory generation method includes: The current environmental features surrounding the autonomous vehicle are acquired; these features are obtained by encoding and fusing current environmental information, which includes image information and map information. Using the current environmental features as anchor point trajectory generation conditions, a multi-anchor point candidate trajectory is generated through a basic trajectory generator; The candidate multi-anchor-point trajectories are superimposed with the first multiplicative noise to obtain the multi-anchor-point exploration trajectory; and the multi-anchor-point exploration trajectory is smoothed and filtered to obtain the smooth exploration trajectory. According to the first preset anchor point, the smooth exploration trajectory is sampled to obtain multiple trajectory variant groups; the intra-group reward score of each trajectory variant in each trajectory variant group is calculated; the trajectory variant with the maximum intra-group reward score in each trajectory variant group is selected as the filtered trajectory variant. Based on the truncated GRPO model between anchor points, a global safety assessment is performed on each selected trajectory variant to obtain the global safety assessment result for each selected trajectory variant; for each selected trajectory variant, the intra-group reward score is corrected based on the global safety assessment result to obtain the global reward score for each selected trajectory variant; and based on the truncated GRPO model between anchor points, a global safety assessment is performed on the smooth exploration trajectory to obtain the global reward score for the smooth exploration trajectory. Based on the global reward score of each selected trajectory variant and the global reward score of the smooth exploration trajectory, all selected trajectory variants and the smooth exploration trajectory are filtered to obtain the optimal driving trajectory.

2. The method for generating truncated diffusion model trajectories based on hierarchical GRPO according to claim 1, characterized in that, The process of calculating the in-group reward score for each trajectory variant within each trajectory variant group includes: Obtain the efficiency bonus, comfort bonus, and rule compliance bonus for each trajectory variant within each trajectory variant group; Grouping by each trajectory variant, the efficiency reward, comfort reward, and rule compliance reward for each trajectory variant are linearly weighted and summed to obtain the comprehensive reward score for each trajectory variant within each trajectory variant group. Based on the comprehensive reward score of each trajectory variant within each trajectory variant group, calculate the mean and standard deviation of the comprehensive reward score within each trajectory variant group; and based on the mean and standard deviation of the comprehensive reward score within each trajectory variant group, normalize the comprehensive reward score of each trajectory variant within the corresponding group to obtain the in-group reward score of each trajectory variant within each trajectory variant group.

3. The method for generating truncated diffusion model trajectories based on hierarchical GRPO according to claim 1 or 2, characterized in that, For each selected trajectory variant, the process of adjusting the group's reward score based on the global security assessment results to obtain the global reward score for each selected trajectory variant includes: For each selected trajectory variant, if the global safety assessment result indicates a collision, the intra-group reward score is corrected to a preset negative value; if the global safety assessment result indicates no collision and the intra-group reward score is less than a preset score threshold, the intra-group reward score is corrected to the preset score threshold; if the global safety assessment result indicates no collision and the intra-group reward score is greater than or equal to the preset score threshold, the intra-group reward score is not corrected. For each filtered trajectory variant, the preset negative value, the preset scoring threshold, or the intra-group reward score is used as the global reward score.

4. The method for generating truncated diffusion model trajectories based on hierarchical GRPO according to claim 1 or 2, characterized in that, The process of filtering all trajectory variants and the smooth exploration trajectory based on the global reward score of each selected trajectory variant and the global reward score of the smooth exploration trajectory to obtain the optimal driving trajectory includes: All the selected trajectory variants and the smoothed exploration trajectory are merged to form a candidate trajectory set; If there is a trajectory in the candidate trajectory set with a preset negative global reward score, then the trajectory with the preset negative global reward score is removed from the candidate trajectory set to obtain the optimized trajectory set. If there is no trajectory in the candidate trajectory set with a preset negative global reward score, then the candidate trajectory set will be used as the optimized trajectory set. The trajectory with the highest global reward score in the optimized trajectory set is taken as the optimal driving trajectory.

5. The method for generating truncated diffusion model trajectories based on hierarchical GRPO according to claim 1 or 2, characterized in that, The process of pre-constructing the truncated diffusion model of the hierarchical GRPO includes: Acquire the first historical environmental features around the autonomous vehicle, and the first historical trajectory information of the autonomous vehicle; The first historical environmental features are input into the pre-trained anchor point diffusion model to obtain a multi-anchor point predicted trajectory; the pre-trained anchor point diffusion model is adjusted by parameters to obtain the basic trajectory generator. The multi-anchor predicted trajectory is superimposed with the second multiplicative noise to obtain the sample exploration trajectory; and the sample exploration trajectory is smoothed and filtered to obtain the smoothed sample trajectory. According to the first preset anchor point, the smooth sample trajectory is sampled to obtain multiple sample trajectory variant groups; the intra-group reward score of each sample trajectory variant in each sample trajectory variant group is calculated; the sample trajectory variant with the maximum intra-group reward score in each sample trajectory variant group is selected as the sample trajectory variant. Based on the preset anchor-point truncated GRPO model, a global security assessment is performed on each selected sample trajectory variant to obtain the global security assessment result for each selected sample trajectory variant. For each selected sample trajectory variant, the intra-group reward score is corrected based on the global security assessment result to obtain the global reward score for each selected sample trajectory variant. Furthermore, based on the preset anchor-point truncated GRPO model, a global security assessment is performed on the smoothed sample trajectory to obtain the global reward score for the smoothed sample trajectory. The preset anchor-point truncated GRPO model is then adjusted to obtain the anchor-point truncated GRPO model. Based on the global reward score of the smoothed sample trajectory, the global reward score of each selected sample trajectory variant, and the first historical trajectory information, a loss function is constructed. With the goal of minimizing the loss function, the parameters in the pre-defined hierarchical GRPO truncated diffusion model are iteratively updated to obtain the hierarchical GRPO truncated diffusion model.

6. The method for generating truncated diffusion model trajectories based on hierarchical GRPO according to claim 5, characterized in that, The expression for the loss function includes: ， in, Represents the loss function. This represents the reinforcement learning loss function. Indicates the weighting coefficient. This represents the loss function for imitation learning; The expression for the reinforcement learning loss function includes: ， in, This represents the expectation, used to calculate the mean of the product of the global reward score and the probability for all trajectories. Representing the trajectory Global reward score, Indicates the current policy model Generate trajectory The probability of; The expression for the imitation learning loss function includes: ， in, Indicates the total number of trajectories. Denotes KL divergence, Indicating the trajectory in the reference strategy The probability distribution is obtained by cloning and training the reference strategy using the first historical trajectory information. Indicates the current policy model Trajectory The predicted probability distribution.

7. The method for generating truncated diffusion model trajectories based on hierarchical GRPO according to claim 5, characterized in that, Before inputting the first historical environmental features into the pre-trained anchor diffusion model, the trajectory generation method further includes: Acquire second historical environmental features and second historical trajectory information; the second historical environmental features occurred before the first historical environmental features, and the second historical trajectory information occurred before the first historical trajectory information. Gaussian noise generated in a random manner is superimposed on the historical trajectory information to obtain a noisy trajectory sample; The noisy trajectory sample, the second historical environmental features, and the second preset anchor point are input into the anchor point diffusion model to obtain the predicted noise. With the goal of minimizing the difference between the predicted noise and the Gaussian noise, the parameters in the anchor point diffusion model are adjusted to obtain the pre-trained anchor point diffusion model.

8. The method for generating truncated diffusion model trajectories based on hierarchical GRPO according to claim 1 or 2, characterized in that, The process of obtaining the current environment features by encoding and fusing the current environment information includes: The image information is converted to the coordinate system of the autonomous vehicle using camera extrinsic parameters to obtain image information in the vehicle coordinate system; the map information is converted to the coordinate system of the autonomous vehicle using the positioning information of the autonomous vehicle to obtain map information in the vehicle coordinate system. Image information in the vehicle coordinate system is image encoded to obtain image bird's-eye view features; map information in the vehicle coordinate system is map encoded to obtain map bird's-eye view features; The current environment features are obtained by multi-scale fusion of the image bird's-eye view features and the map bird's-eye view features.

9. The method for generating truncated diffusion model trajectories based on hierarchical GRPO according to claim 1 or 2, characterized in that, The calculation formula for the multi-anchor point exploration trajectory includes: ， in, This indicates the multi-anchor point exploration trajectory in the th... The state of the time step, Indicates the candidate trajectory of multiple anchor points in the th... The state of the time step, Indicates the first The first multiplicative noise at the time step; In the The formula for calculating the first multiplicative noise at the time step includes: ， in, Indicates the first Longitudinal noise at time step Indicates the first Lateral noise at the time step.

10. A trajectory generation system for a truncated diffusion model based on hierarchical GRPO, characterized in that, include: The feature acquisition module is used to acquire the current environmental features around the autonomous vehicle; The current environmental characteristics are obtained by encoding and fusing current environmental information; The current environment information includes: image information and map information; The trajectory generation module is used to generate multi-anchor point candidate trajectories by using the current environmental features as anchor point trajectory generation conditions and through a basic trajectory generator. The trajectory smoothing module is used to superimpose the multi-anchor candidate trajectory with the first multiplicative noise to obtain the multi-anchor exploration trajectory; and to perform smoothing filtering on the multi-anchor exploration trajectory to obtain the smoothed exploration trajectory. The intra-group evaluation module is used to sample the smooth exploration trajectory according to the first preset anchor point to obtain multiple trajectory variant groups; calculate the intra-group reward score of each trajectory variant in each trajectory variant group; and select the trajectory variant with the maximum intra-group reward score in each trajectory variant group as the selected trajectory variant. The global assessment module is used to perform a global safety assessment on each selected trajectory variant based on the anchor-truncation GRPO model, and obtain a global safety assessment result for each selected trajectory variant; for each selected trajectory variant, the intra-group reward score is corrected based on the global safety assessment result, and a global reward score is obtained for each selected trajectory variant; and a global safety assessment is performed on the smooth exploration trajectory based on the anchor-truncation GRPO model, and a global reward score is obtained for the smooth exploration trajectory. The trajectory filtering module is used to filter all trajectory variants and the smooth exploration trajectory based on the global reward score of each filtered trajectory variant and the global reward score of the smooth exploration trajectory to obtain the optimal driving trajectory.

Images