Unmanned vehicle trajectory prediction method based on fuzzy inverse reinforcement learning

Abstract

The application discloses an unmanned vehicle trajectory prediction method based on fuzzy inverse reinforcement learning, which comprises the following steps: calculating trajectory features, introducing fuzzy logic for inverse reinforcement learning to find an optimal weight vector, and optimal trajectory prediction simulation. The application mainly processes unmanned vehicle driving trajectory data and actual data of human drivers, combines fuzzy logic processing, trains an inverse reinforcement learning model, obtains an optimal weight vector, and thus calculates the optimal driving trajectory of the unmanned vehicle in a limited trajectory. The application solves the problems of poor adaptability to complex environments and low decision accuracy of the unmanned vehicle in the driving strategy process in the prior art, and significantly improves the driving strategy efficiency and safety of the unmanned vehicle in the actual road environment. The introduction of the fuzzy inverse reinforcement learning method enables the unmanned vehicle to better learn the driving strategy of human drivers, thereby realizing more intelligent and safe automatic driving functions.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, and particularly relates to an unmanned vehicle trajectory prediction method based on fuzzy inverse reinforcement learning. BACKGROUND

[0002] In recent years, automatic driving technology has gradually become an important part of intelligent transportation systems, aiming to improve the safety and efficiency of road traffic through automation and intelligent means. As an important form of automatic driving technology, one of the cores of unmanned vehicles is to develop accurate and safe driving strategies. Driving trajectory decision, as an operation that must be frequently performed by unmanned vehicles in complex road environments, directly affects the driving safety of vehicles and the smoothness of traffic flow. However, existing driving trajectory decision methods still have problems such as poor adaptability and low decision accuracy when facing complex and dynamic traffic environments. Therefore, how to improve the driving trajectory decision capability of unmanned vehicles in complex road environments has become one of the research hotspots in the field of current automatic driving technology.

[0003] Traditional unmanned vehicle driving strategies mainly include rule-based methods and model-based methods. However, these methods have the following shortcomings:

[0004] 1. Rule-based methods make decisions through predefined rules. This method is simple and easy to implement, but it is difficult to adapt to various unexpected situations and has poor robustness when facing variable and complex traffic environments.

[0005] 2. Model-based methods predict and control the motion state of the vehicle by establishing mathematical models, including trajectory planning-based methods, game theory-based methods, and machine learning-based methods. Although these methods have improved the intelligent level of developing driving strategies to some extent, they still have some limitations. For example, trajectory planning methods require a large amount of computing resources and are difficult to implement in real-time applications; game theory methods rely on accurate prediction of the behavior of other vehicles and are not effective in complex and uncertain traffic environments; and traditional machine learning methods can achieve a certain adaptability through data-driven, but there are still great challenges in feature selection and model training.

[0006] Fuzzy logic is an effective method for handling uncertainty and fuzziness, and has important applications in complex decision-making scenarios. Unlike traditional binary logic, fuzzy logic allows variables to have continuous values between 0 and 1, thereby better describing the uncertainty and fuzziness in the real world. In the driving strategy of unmanned vehicles, a fuzzy logic system can make more flexible and robust decisions based on various uncertain factors of the traffic environment, such as vehicle speed, vehicle distance, road conditions, etc.

[0007] In recent years, with the rapid development of deep learning and reinforcement learning technology, inverse reinforcement learning (IRL) as a method that can obtain the optimal policy through actual learning has gradually attracted the attention of researchers. Inverse reinforcement learning derives the reward function by learning expert actual data, so that the unmanned vehicle can simulate the decision-making behavior of human drivers. However, the traditional inverse reinforcement learning method still faces some problems in practical application. On the one hand, in a complex and uncertain traffic environment, it may lead to poor robustness of decision-making. On the other hand, only optimizing a single target, it is difficult to consider multiple performance indicators at the same time, such as the balance between safety and efficiency, thereby affecting the comprehensive quality of decision-making.

[0008] Therefore, there is an urgent need for an unmanned vehicle trajectory simulation prediction method that can adapt and be efficient in a complex traffic environment to solve the problems in the prior art and provide a safer and more intelligent driving strategy scheme. SUMMARY

[0009] The purpose of the present application is to propose an unmanned vehicle trajectory prediction method based on fuzzy inverse reinforcement learning, which fully utilizes the experience data of human drivers by combining fuzzy logic and inverse reinforcement learning, and realizes efficient adaptation to complex traffic environments. This method not only solves the problems of strong dependence on pre-defined rules, large demand for labeled data and high computational complexity in traditional methods, effectively solves the problems of poor adaptability and low decision-making accuracy of unmanned vehicles in complex traffic environments when formulating driving strategies, but also significantly improves the decision-making accuracy and robustness of unmanned vehicles in dynamic environments.

[0010] The present application relates to the technical field of autonomous driving, in particular to an unmanned vehicle trajectory prediction method based on fuzzy inverse reinforcement learning. The method solves the problem of formulating driving strategies for unmanned vehicles in complex traffic environments by introducing a combination of fuzzy logic and inverse reinforcement learning, and improves the accuracy and robustness of formulating driving strategies.

[0011] The specific technical solution to achieve the purpose of the present application is:

[0012] An unmanned vehicle trajectory prediction method based on fuzzy inverse reinforcement learning, comprising the following steps:

[0013] Step 1: Calculate the trajectory features, including extracting the vehicle data in the NGSIM dataset, denoising and normalizing the vehicle data, randomly extracting the predicted trajectory, calculating the position and speed of the preceding vehicle at the initial position of the predicted trajectory, the position and speed of the preceding and following vehicles at the end position of the trajectory, generating a limited number of predicted trajectories with the same initial position, and calculating the feature vector t exp of the predicted trajectory h ;

[0014] Step two: introducing fuzzy logic to inverse reinforcement learning to find the optimal weight vector θ * , including initializing a random weight vector θ0, iterating N times for a limited number of predicted trajectories, in each iteration, initializing the feature expectation value f exp of the human real trajectory and the feature expectation value f h of the predicted trajectory, and traversing each predicted trajectory corresponding to the human real trajectory randomly extracted in step one, calculating the reward r = θ T · t of each trajectory according to the transpose vector θ T of the current weight vector θ and the trajectory feature t, calculating the occurrence probability p of each trajectory according to the reward r; then, calculating the predicted trajectory feature matrix T = p·t and the human real trajectory feature matrix H = t h using the occurrence probability p and the trajectory feature t, updating the predicted trajectory feature expectation value f exp + = T and the human real trajectory feature expectation value f h + = H, after traversing all predicted trajectories, fuzzy processing the human real trajectory feature expectation value f' exp = f exp · u, fuzzy processing the predicted trajectory feature expectation value f' h = f h · u, then, calculating the gradient g = f' h - f' exp - λθ, and then calculating the combined gradient g comb = αg - βθ, wherein the hyperparameter α is used to adjust the gradient, and the hyperparameter β is used to adjust the weight update, finally, using the combined gradient to calculate the latest weight vector θ' = θ + lr·g comb , bringing the latest weight vector θ' into the next iteration, repeating this process until the iteration is completed, and obtaining the final weight vector θ * ;

[0015] Step three: trajectory simulation prediction, including generating a limited number of predicted trajectories by a given action combination and calculating the trajectory feature, using the final weight vector θ * generated in step two to calculate the reward r of each predicted trajectory, and then calculating the occurrence probability p from the reward value, finally, selecting the trajectory with the highest occurrence probability as the optimal trajectory for the vehicle to travel.

[0016] Further, the step one specifically includes the following steps:

[0017] Step A1: Select data in a specific time period containing five lanes from the NGSIM dataset, the selected dataset covers a one-hour time range, preprocess the extracted raw data, use a filter to remove noise and outliers in the data, normalize the denoised data, and convert different dimensional data to a unified scale;

[0018] Step A2: Randomly select several prediction trajectories from the dataset, for each prediction trajectory, calculate the position and speed of the preceding vehicle at the initial position, calculate the position and speed information of the preceding and following vehicles at the end position, use the initial lateral coordinate x0, longitudinal coordinate y0, longitudinal speed v y0 , longitudinal acceleration a y , target position lateral coordinate x', running time t to generate N prediction trajectory sets Where i represents the ith trajectory ζ i , each trajectory ζ starts from the same initial position with the same longitudinal speed and longitudinal acceleration;

[0019] Step A3: Calculate the feature vector t exp of the prediction trajectory and the feature vector t h of the human real trajectory, which includes the current vehicle features: position, speed, acceleration, longitudinal acceleration absolute value, lateral acceleration absolute value, longitudinal jerk absolute value, and the features of the preceding vehicle at the starting position, the features of the preceding and following vehicles at the target position: position and speed of the preceding and following vehicles, acceleration of the preceding and following vehicles, safety distance of the preceding and following vehicles, and the acceleration of all vehicles at the current time.

[0020] Further, the step two specifically comprises the following steps:

[0021] Step B1: At the beginning of the training process, first initialize a random weight vector θ0, which will be used to calculate the trajectory features and rewards in the subsequent steps;

[0022] Step B2: Find the optimal weight θ iters through N * iterations of a limited number of prediction trajectories, in each iteration, initialize the feature expectation value f exp of the human real trajectory and the feature expectation value f h of the prediction trajectory; traverse the human real trajectory randomly extracted in step one, calculate the reward r for each prediction trajectory according to the current weight vector θ and the trajectory feature t, then calculate the probability p of each trajectory; use these probabilities p and trajectory features t to calculate the prediction trajectory feature matrix T and the human real trajectory feature matrix H, and update the prediction trajectory feature expectation value f exp and the human trajectory feature expectation value f h ;

[0023] Step B3: generate a blur factor u for each feature, which fluctuates randomly within a predetermined range; specifically, the blur factor u is a random number uniformly distributed in the range [1-δ, 1+δ], and the δ parameter controls the fluctuation range of the blur factor;

[0024] Step B4: apply the generated blur factor u to the predicted trajectory feature expectation value f exp and the human trajectory feature expectation value f h , that is, multiply the expectation value f of each feature by the corresponding blur factor u;

[0025] Step B5: after introducing the blur factor, perform gradient g calculation; the gradient g is a vector representing the difference between the human feature expectation value and the generated feature expectation value, and subtracts the regularization term λθ, which points to the direction in which the function value increases fastest;

[0026] Step B6: combine the gradient g comb to obtain a new gradient g comb by weighted combination of the original gradient g and the weight vector θ;

[0027] Step B7: use the learning rate lr and the combined gradient g comb to calculate the latest weight vector θ' = θ + lr·g comb ;

[0028] Step B8: determine whether N iters iterations are completed, if so, the final weight vector θ * is equal to the weight vector θ' calculated in the last iteration, that is, θ * = θ'.

[0029] Further, the step three specifically includes the following steps:

[0030] Step C1: generate multiple predicted trajectories by combining given actions, and perform feature calculation on these trajectories to ensure coverage of possible driving paths;

[0031] Step C2: use the final weight vector obtained in step two to perform reward calculation on each predicted trajectory, and then calculate the occurrence probability of each trajectory according to these reward values, which represents the possibility of being the actual driving path;

[0032] Step C3: select the trajectory with the highest occurrence probability from all calculated trajectories as the optimal trajectory for the vehicle to drive.

[0033] Through the above steps, the method of the present application has the following beneficial effects:

[0034] 1. Improve decision accuracy: the combination of fuzzy logic processing and inverse reinforcement learning model improves the accuracy of the unmanned vehicle in formulating driving strategies in complex traffic environments;

[0035] 2. Enhanced robustness: The introduction of fuzzy logic makes the decision-making process more robust, effectively dealing with uncertainties in the traffic environment;

[0036] 3. Reduced computational complexity: Compared with traditional reinforcement learning methods, the method of the present application significantly reduces the computational complexity while ensuring efficiency, improving the speed of model training and inference;

[0037] 4. Strong practicality: The verification results in the simulation environment show that the method can develop effective driving strategies in actual traffic scenarios, and has high practical value. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 is a flowchart of the present application;

[0039] Figure 2 is a flowchart of the present application for calculating trajectory features;

[0040] Figure 3 is a flowchart of the present application for introducing fuzzy logic for inverse reinforcement learning to find the optimal weight vector;

[0041] Figure 4 is a flowchart of the present application for optimal trajectory prediction simulation;

[0042] Figure 5 is a simulation diagram of the unmanned vehicle driving trajectory simulation prediction method;

[0043] Figure 6 is a line graph of the combined loss varying with the number of iterations during model training. DETAILED DESCRIPTION

[0044] The present application will be further described in detail in conjunction with the following specific examples and drawings. The process, conditions, experimental methods, etc. for implementing the present application are generally known and commonly understood, and the present application does not have special restrictions.

[0045] As shown in Figure 1 , the method of the present application comprises the following steps:

[0046] Step 1: Calculate the trajectory feature, including extracting the vehicle data in the NGSIM data set, and denoising and normalizing the vehicle, randomly extracting the predicted trajectory, calculating the position and speed of the front vehicle at the initial position of the trajectory, the position and speed of the front and rear vehicles at the end position of the trajectory, generating a limited number of predicted trajectories with the same initial position, and calculating the feature vector t exp of the predicted trajectory and the feature vector t h of the human real trajectory;

[0047] Step 2: Introduce fuzzy logic for inverse reinforcement learning to find the optimal weight vector θ * It involves initializing a random weight vector θ0, performing N iterations on a finite number of predicted trajectories, and in each iteration, initializing the expected value f of the features of the actual human trajectory. exp and the expected value f of the predicted trajectory h It iterates through each predicted trajectory corresponding to the real human trajectory in step one, and then uses the transpose of the current weight vector θ to calculate the predicted trajectory. T The reward r = θ for each trajectory is calculated from the trajectory feature t. T Based on the reward r, the probability p of each trajectory is calculated. Then, the predicted trajectory feature matrix T = p·t and the human true trajectory feature matrix H = t are calculated using the probability p and the trajectory feature t. h Used to update the expected value f of the predicted trajectory features exp +=T and expected value of human trajectory features f h +=H, after traversing all predicted trajectories, the expected value of human features is fuzzy processed by f'. exp =f exp •u, fuzzy processing is performed on the expected value of the predicted features f' h =f h ·u, then calculate the gradient g=f' h -f' exp -λθ, then calculate the combined gradient g comb =αg-βθ, where the hyperparameter α is used to adjust the gradient, the hyperparameter β is used to adjust the weight update, and finally, the combined gradient is used to calculate the latest weight vector θ' = θ + lr·g comb The latest weight vector θ' is then substituted into the next iteration, and this process is repeated until the iteration ends, yielding the final weight vector θ. * ;

[0048] Step 3: Trajectory simulation and prediction, which involves generating a finite number of predicted trajectories based on a given combination of actions and calculating trajectory features, using the weight vector θ generated in Step 2. * The reward r for each predicted trajectory is calculated, and the probability p of occurrence is calculated from the reward value. Finally, the trajectory with the highest probability of occurrence is selected as the optimal trajectory for the vehicle.

[0049] like Figure 2 As shown, step one includes calculating trajectory features, which involves extracting vehicle data from the NGSIM dataset, performing denoising and normalization on the vehicles, randomly extracting predicted trajectories, calculating the position and speed of the preceding vehicle at the initial position of the trajectory, and the position and speed information of the preceding and following vehicles at the end position of the trajectory, generating a finite number of predicted trajectories with the same initial position, and calculating the feature vector t of the predicted trajectory.exp The feature vector t of the actual human trajectory h This includes the following steps:

[0050] Step A1: Select data from the NGSIM dataset that includes five lanes within a specific time period. The selected dataset covers a one-hour time range. Preprocess the extracted raw data by using filters to remove noise and outliers from the data. Normalize the denoised data to convert data of different dimensions into a uniform scale.

[0051] Step A2: Randomly select several predicted trajectories from the dataset. For each predicted trajectory, calculate the position and velocity of the vehicle in front at its initial position, and calculate the position and velocity information of the vehicle in front and behind at its final position, using the vehicle's initial lateral coordinate x0, longitudinal coordinate y0, and longitudinal velocity v. y0 Longitudinal acceleration a y The target location's lateral coordinate x' and the running time t generate N sets of predicted trajectories. Where i represents the i-th trajectory ζ i Each trajectory ζ starts from the same initial position with the same longitudinal velocity and longitudinal acceleration;

[0052] Step A3: Calculate the feature vector t of the predicted trajectory exp The feature vector t of the actual human trajectory h It includes the current vehicle characteristics: position, speed, acceleration, absolute value of longitudinal acceleration, absolute value of lateral acceleration, absolute value of longitudinal acceleration, and characteristics of the vehicle in front at the starting position and the vehicles in front and behind at the target position: position and speed of the vehicles in front and behind, acceleration of the vehicles in front and behind, safe distance between the vehicles in front and behind, speed difference between the vehicles in front and behind, and acceleration of all vehicles at the current moment.

[0053] like Figure 3 As shown, step two includes introducing fuzzy logic to perform inverse reinforcement learning in order to find the optimal weight vector θ. * It involves initializing a random weight vector θ0, performing N iterations on a finite number of predicted trajectories, and in each iteration, initializing the expected value f of the features of the actual human trajectory. exp and the expected value f of the predicted trajectory h It iterates through each predicted trajectory corresponding to the real human trajectory in step one, and then uses the transpose of the current weight vector θ to calculate the predicted trajectory. T The reward r = θ for each trajectory is calculated from the trajectory feature t. T Based on the reward r, the probability p of each trajectory is calculated. Then, the predicted trajectory feature matrix T = p·t and the human true trajectory feature matrix H = t are calculated using the probability p and the trajectory feature t. h Used to update the expected value f of the predicted trajectory featuresexp += T and human trajectory feature expectation f h += H, after traversing all predicted trajectories, blur the human feature expectation f exp = f exp · u, blur the predicted feature expectation f h = f h · u, then calculate the gradient g = f h - f exp - ltheta, recalculate the combined gradient g comb = alpha g - beta theta, where the hyperparameter alpha is used to adjust the gradient, the hyperparameter beta is used to adjust the weight update, and finally, the latest weight vector theta' = theta + lr · g is calculated using the combined gradient comb , the latest weight vector theta' is brought into the next iteration, and this is repeated until the iteration ends, and the final weight vector theta is obtained * . The steps include the following:

[0054] Step B1: At the beginning of the training process, first initialize a random weight vector theta0, which will be used to calculate the trajectory feature and reward in the subsequent steps;

[0055] Step B2: and find the optimal weight theta iters through N * iterations of limited predicted trajectories exp , in each iteration, initialize the feature expectation f h of the human real trajectory and the feature expectation f exp of the predicted trajectory h . Traverse the human real trajectory randomly extracted in step one, calculate the reward r for each predicted trajectory according to the current weight vector theta and the trajectory feature t, and then calculate the probability p of each trajectory. Use these probabilities p and trajectory features t to calculate the predicted trajectory feature matrix T and the human real trajectory feature matrix H, and update the predicted trajectory feature expectation f exp and the human trajectory feature expectation f exp ;

[0056] 1) Initialize a zero vector of size F for the predicted trajectory feature expectation f F = 0 h ;

[0057] 2) Initialize a zero vector of size F for the human trajectory feature expectation f h = 0 F ;

[0058] 3) Initialize trajectory index counter i: i = 0, since all the predicted trajectory features need to be processed one by one, the index counter is used to track the index of the predicted trajectory that is currently being processed, indicating that the ith predicted trajectory ζ i is currently being calculated; 4) Calculate the reward value r i of the trajectory feature vector t i under the given parameter vector θ:

[0059] r i = θ T · t i

[0060] where θ T represents the transpose of the θ vector;

[0061] 5) Calculate the normalized probability: calculate the normalized occurrence probability p i , to ensure that the occurrence probability of each trajectory is within the range [0, 1], and the sum of the probabilities of all trajectories is 1;

[0062]

[0063] where the minus max({r j}) operation is to prevent the problem of exponential overflow in calculation, to ensure the stability of calculation. Here i is the index of the element currently being calculated, and j is the index of all elements used to calculate the normalization term (denominator);

[0064] 6) Calculate the trajectory feature matrix: calculate the weighted sum of the trajectory feature matrix T, which combines the feature information t i of all trajectories, weighted according to their occurrence probability p i ;

[0065]

[0066] T represents the weighted sum of all trajectory features, with the weight being the occurrence probability of the trajectory;

[0067] 7) Update the human trajectory feature expectation f exp by adding the feature matrix T under the current scenario:

[0068] f+ = T

[0069] exp

[0070] The predicted trajectory feature expectation is a reflection of the average feature of the model's predicted trajectory. In each iteration, by updating the predicted trajectory feature expectation f exp , the model gradually adjusts its predicted trajectory features to be closer to the target (i.e., the human trajectory feature expectation);

[0071] 8) Since the occurrence probability of the human true trajectory is 100%, the human actual vehicle trajectory feature matrix H is equal to the current trajectory ζ h h ;

[0072] H = t h

[0073] 9) Update the human trajectory feature expectation value:

[0074] f h + = H

[0075] By updating the human trajectory feature expectation value, the target feature can be continuously corrected to better reflect the actual situation of human driving behavior, thereby improving the learning effect of the model;

[0076] 10) The index counter value is incremented by one, and the index counter is incremented by one every time a predicted trajectory feature is processed, pointing to the next predicted trajectory to be processed:

[0077] i + = 1

[0078] Step B3: Generate a fuzzy factor u for each feature, which randomly fluctuates within a predetermined range. Specifically, the fuzzy factor u is a random number uniformly distributed in the range [1-δ, 1+δ], and the δ parameter controls the fluctuation range of the fuzzy factor;

[0079] u = Uniform(1-δ, 1+δ, F)

[0080] Step B4: Apply the generated fuzzy factor u to the predicted trajectory feature expectation value f exp and the human trajectory feature expectation value f h , that is, multiply the expected value f of each feature by the corresponding fuzzy factor u;

[0081] 1) Apply the fuzzy factor to calculate the predicted trajectory feature expectation value: apply the fuzzy factor u to the predicted trajectory feature expectation value to simulate the uncertainty in the actual environment.

[0082] f' = f · u

[0083] exp exp

[0084] 2) Apply the fuzzy factor to calculate the human trajectory feature expectation value: apply the fuzzy factor u to the human trajectory feature expectation value to simulate the uncertainty in the actual environment;

[0085] f' h = f h · u

[0086] ​Step B5: Gradient g calculation after introducing blur factor; Gradient g is a vector representing the difference between human feature expectation value and generated feature expectation value, and subtracting regularization term λθ, pointing to the direction of fastest function value increase;

[0087] g = f' h -f' exp -λθ

[0088] Wherein, the regularization parameter λ imposes a constraint on the weight vector θ in the formula for calculating the gradient, thereby preventing overfitting and controlling model complexity;

[0089] Step B6: Combine gradient g comb is obtained by weighted combination of original gradient g and weight vector θ. This combined gradient can more accurately reflect the direction and amplitude that need to be adjusted, in order to optimize the weight vector more effectively;

[0090] g comb = αg - βθ

[0091] Wherein, α is the hyperparameter for adjusting the gradient, and β is the hyperparameter for adjusting the weight update, the numerical value is determined through experiments, try multiple values, through observation of the training process and performance of the model, select the most appropriate value;

[0092] Step B7: Use learning rate lr and combined gradient g comb to calculate the latest weight vector θ';

[0093] θ' = θ + lr·g comb

[0094] Wherein, lr is the learning rate, used to control the step size of parameter update, as shown in Figure 6 , which shows the change of combined loss L comb = α·||θ'|| + β·||f' h -f' exp || during the training process, which decreases with the increase of iteration number, indicating that the performance of the model is gradually improved, and the generated loss α·||θ'|| and the regularization loss β·||f' h -f' exp || are gradually converging;

[0095] Step B8: Determine whether N iters times of iteration are completed, if completed, the final weight vector θ * is equal to the weight vector θ' calculated last time;

[0096] θ * = θ'

[0097] As Figure 4As shown, step three includes testing and validating the optimal driving strategy in a simulation environment to ensure its effectiveness and safety in complex traffic environments, including the following steps:

[0098] Step C1: Generate multiple predicted trajectories based on the given action combinations, and perform feature calculations on these trajectories to ensure coverage of possible driving paths;

[0099] Step C2: Using the final weight vector obtained in Step 2, calculate the reward for each predicted trajectory, and then calculate the probability of each trajectory occurring based on these reward values, representing its likelihood as an actual driving path;

[0100] like Figure 5 As shown, step C3: Select the trajectory with the highest probability of occurrence from the obtained trajectories as the optimal trajectory for vehicle travel.

[0101] The scope of protection of this invention is not limited to the above embodiments. Any variations and advantages that can be conceived by those skilled in the art without departing from the spirit and scope of the inventive concept are included in this invention and are protected by the appended claims.

Claims

1. An unmanned vehicle trajectory prediction method based on fuzzy inverse reinforcement learning, characterized in that, The method comprises the following steps: Step one: calculate the trajectory feature, including extracting the vehicle data in the NGSIM dataset, and denoising and normalizing the vehicle data, randomly extracting the predicted trajectory, calculating the position and speed of the front vehicle at the initial position of the predicted trajectory, the position and speed of the front and rear vehicles at the end position of the trajectory, generating a limited number of predicted trajectories with the same initial position, and calculating the feature vector t of the predicted trajectory exp and the feature vector t of the real trajectory of the human h ; Step two: introducing fuzzy logic to inverse reinforcement learning to find the optimal weight vector θ * , including initializing a random weight vector θ0, iterating N times for a limited number of predicted trajectories, initializing the feature expectation value f exp of the human real trajectory and the feature expectation value f h of the predicted trajectory in each iteration, and traversing each predicted trajectory corresponding to the human real trajectory randomly extracted in step one, calculating the reward r = θ T · t of each trajectory according to the transpose vector θ T of the current weight vector θ and the trajectory feature t, calculating the occurrence probability p of each trajectory according to the reward r; then, calculating the predicted trajectory feature matrix T = p·t and the human real trajectory feature matrix H = t h using the occurrence probability p and the trajectory feature t, updating the predicted trajectory feature expectation value f exp + = T and the human real trajectory feature expectation value f h + = H, fuzzy processing the human real trajectory feature expectation value f' exp = f exp · u, and fuzzy processing the predicted trajectory feature expectation value f' h = f h · u, u being the fuzzy factor; then, calculating the gradient g = f' h - f' exp - λθ, and then calculating the combined gradient g comb = αg - βθ, where the hyperparameter α is used to adjust the gradient and the hyperparameter β is used to adjust the weight update, finally, using the combined gradient to calculate the latest weight vector θ' = θ + lr·g comb , bringing the latest weight vector θ' into the next iteration, repeating the above process until the iteration ends, and obtaining the final weight vector θ * ; Step three: trajectory simulation prediction, including generating a limited number of prediction trajectories by given action combination and calculating trajectory features, using the final generated weight vector θ in step two * , calculating the reward r of each prediction trajectory, then calculating the occurrence probability p from the reward value, finally, selecting the trajectory with the highest occurrence probability as the optimal trajectory for the vehicle to travel.

2. The method of claim 1, wherein, The first step specifically comprises the following steps: Step A1: Select data in a specific time period containing five lanes from the NGSIM dataset, the selected dataset covers a one-hour time range, preprocess the extracted raw data, use a filter to remove noise and outliers in the data, and normalize the denoised data to convert different dimensional data to a unified scale; Step A2: randomly select several predicted trajectories from the dataset, for each predicted trajectory, calculate the front vehicle position and speed at its initial position, calculate the position and speed information of the front and rear vehicles at its end position, use the initial lateral coordinate x0, longitudinal coordinate y0, longitudinal speed v y0 , longitudinal acceleration a y of the vehicle, the lateral coordinate x' of the target position, and the running time t to generate a set of N predicted trajectories where i represents the ith trajectory ζ i , each trajectory ζ starts from the same initial position with the same longitudinal speed and longitudinal acceleration; Step A3: Calculate the feature vector t of the predicted trajectory exp and the feature vector t of the real trajectory of the human h , including the current vehicle features: position, speed, acceleration, longitudinal acceleration absolute value, lateral acceleration absolute value, longitudinal jerk absolute value, and the features of the front and rear vehicles before the starting position and the target position: the positions and speeds of the front and rear vehicles, the accelerations of the front and rear vehicles, the safety distances of the front and rear vehicles, the speed differences of the front and rear vehicles, and the accelerations of all vehicles at the current time.

3. The method of claim 1, wherein, The second step specifically comprises the following steps: Step B1: At the beginning of the training process, first initialize a random weight vector θ0, which will be used to calculate the trajectory features and rewards in the subsequent steps; Step B2: and by N iters Optimal weights θ are found by iteratively refining the finite strip prediction trajectories * In each iteration, the feature expectation value f exp of the human real trajectories is initialized h The human real trajectories randomly extracted in step one are traversed, and for each prediction trajectory, the reward r is calculated according to the current weight vector θ and the trajectory feature t, then the probability p of each trajectory is calculated. The prediction trajectory feature matrix T and the human real trajectory feature matrix H are calculated using these probabilities p and trajectory features t, and the prediction trajectory feature expectation value f exp and the human trajectory feature expectation value f h are updated accordingly. Step B3: Generate a fuzzy factor u for each feature, the fuzzy factor u is a random number uniformly distributed in the range [1-δ, 1+δ], and the δ parameter controls the fluctuation range of the fuzzy factor; Step B4: Apply the generated blur factor u to the predicted trajectory feature expectation f exp and the human trajectory feature expectation f h which is to multiply the expectation f of each feature by the corresponding blur factor u; Step B5: After introducing the fuzzy factor, the gradient g is calculated; the gradient g is a vector representing the difference between the human feature expectation value and the generated feature expectation value, and subtracting the regularization term λθ, pointing to the direction of the fastest function value increase; Step B6: Combining the gradients g comb is obtained by combining the original gradients g and the weight vector θ; Step B7: Use learning rate lr and combined gradient g comb Compute new weight vector θ' = θ + lr · g comb ; Step B8: Determine N iters If the iteration is complete, then the final weight vector θ is determined. * It equals the weight vector θ' calculated in the last calculation, i.e., θ * =θ'.

4. The method of claim 1, wherein, The third step specifically comprises the following steps: Step C1: Generate multiple predicted trajectories through the given action combination, and calculate the features of these trajectories to ensure coverage of possible driving paths; Step C2: Use the maximum weight vector obtained in step two to calculate the rewards of each predicted trajectory, then calculate the occurrence probability of each trajectory according to these reward values, which represents the possibility of it as an actual driving path; Step C3: Select the trajectory with the highest occurrence probability from all calculated trajectories as the optimal trajectory for vehicle driving.

Images