Intelligent driving planning method for pedestrian interaction scene

Abstract

The invention discloses an intelligent driving planning method for a pedestrian interaction scene, and belongs to the technical field of intelligent driving, and the method comprises the steps: building a park environment pedestrian interaction driving scene, and defining the observation and perception states of a vehicle sensor, a target function, and a bottom layer transverse and longitudinal controller; initializing model parameters and an experience playback pool; a human expert drives a vehicle in the simulation scene to generate a driving data sequence, and the driving data sequence is stored in an experience playback pool; training the model by using data in the experience playback pool; deploying a trained model in a simulation scene, planning a vehicle running track, and using a vehicle bottom layer transverse and longitudinal controller to control the vehicle to run according to the planned track; judging whether the model performance reaches the standard or not, and if not, continuing training; and obtaining the model updated at the last time as a training result. According to the invention, the safety and operation efficiency of the intelligent driving planning task in the pedestrian interaction scene can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving technology, and in particular to an intelligent driving planning method for pedestrian interaction scenarios. Background Art

[0002] As an important part of future transportation, the development of autonomous vehicles not only heralds a significant improvement in traffic safety and efficiency, but also represents the rise of sustainable development and new business models. In this process, intelligent driving planning technology plays a vital role, especially in complex high-interaction driving scenarios, a typical example of which is the pedestrian interaction scenario in the park. Traditional rule-based methods, such as state machines and decision trees, can provide effective solutions in specific situations, but they are limited in their ability to handle large-scale state spaces and dynamically changing traffic environments.

[0003] Model predictive control (MPC), as an advanced control strategy, is increasingly being used in intelligent driving planning for autonomous vehicles. MPC establishes a dynamic model of the vehicle, optimizes the driving trajectory in real time, and takes into account the vehicle's dynamic constraints and real-time perceived environmental information, thereby achieving precise control of the vehicle's motion. However, traditional MPC has many defects in the path planning task of intelligent driving. For example, the high computational complexity of MPC leads to poor real-time performance, especially in high-precision control problems; secondly, MPC requires the design of an accurate system model for prediction and control, which means that a high level of system modeling capability is required. However, considering the uncertainty of pedestrian movement in the real environment and the lack of high-precision maps, it is actually almost impossible to accurately design a system model of the driving environment in which the car is located, which ultimately leads to low decision-making performance. Summary of the invention

[0004] The present invention provides an intelligent driving planning method for pedestrian interaction scenarios to solve the technical problems of poor trajectory quality, low interactivity and lack of anthropomorphism in existing intelligent driving planning methods in complex interaction scenarios.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0006] On the one hand, the present invention provides an intelligent driving planning method for pedestrian interaction scenarios, comprising:

[0007] S1, build a pedestrian interactive driving scenario in a campus environment, and define vehicle sensor observation, vehicle perception status, objective function, and vehicle underlying lateral and longitudinal controllers;

[0008] S2, initialize the neural network model parameters and the experience replay pool; the neural network model is used to generate candidate planning trajectories; the experience replay pool is used to store driving data sequences of human experts, each of which consists of vehicle sensor observations, vehicle perception states, and objective function values ​​at multiple moments;

[0009] S3, human experts drive vehicles in the pedestrian interaction driving scenario in the campus environment, complete vehicle navigation tasks under various pedestrian densities and various road conditions, generate corresponding driving data sequences, and store them in the experience replay pool;

[0010] S4, training the neural network model using the driving data sequence in the experience replay pool until a preset training termination condition is reached;

[0011] S5, deploys the trained neural network model in the pedestrian interactive driving scenario in the campus environment, plans the vehicle's running trajectory, and uses the vehicle's underlying lateral and longitudinal controllers to control the vehicle to run according to the planned trajectory;

[0012] S6, judging whether the performance of the neural network model meets the standard. If the performance of the neural network model does not meet the standard in a certain scenario, more driving data sequences in the scenario that does not meet the standard are collected by human experts, stored in the experience replay pool, and S4 to S5 are repeated until the performance of the neural network model meets the standard in all scenarios;

[0013] S7, the training is finished, and the last updated neural network model is obtained as the training result.

[0014] Furthermore, the construction of a pedestrian interactive driving scene in a park environment includes:

[0015] Based on the Carla simulator, a pedestrian interactive driving scenario in a campus environment is built. According to the actual campus scenario, the corresponding OpenDRIVE format map is constructed, and multiple pedestrians with different shapes and behaviors are randomly generated. The starting position and target position are randomly assigned to these pedestrians so that they walk along possible paths. When the pedestrians reach the predetermined target position, a new target position is assigned to them, so that the generated pedestrians move in the campus.

[0016] Further, the vehicle sensor observation includes image information collected by the vehicle surround view camera and point cloud information collected by the vehicle laser radar; wherein the point cloud information collected by the vehicle laser radar is converted into a bird's-eye view with a fixed resolution, forming a square grid centered on the vehicle with a preset length as the side length, dividing the grid into a plurality of grid blocks of the same size, and quantizing the height information of each grid block into a preset number of levels;

[0017] The vehicle perception state includes the vehicle motion state, the pedestrian motion state and the lane line position state; wherein the vehicle motion state includes the vehicle's global coordinates, speed, acceleration, direction and the width and length of the vehicle body; the pedestrian motion state includes the pedestrian's global coordinates, speed, acceleration and direction; the lane line position state includes multiple two-dimensional coordinate points of each lane line;

[0018] The objective function is expressed as:

[0019] c=α1c safe +α2c speed +α3c lane +α4c comfort

[0020] Wherein, c represents the objective function; α1, α2, α3, α4 are preset weighting coefficients; Among them, N obs is the number of pedestrians, d safe is the preset safety distance, d i is the distance between the ith pedestrian and the vehicle; c speed =β1v car +β2a car , where v car is the speed of the vehicle, a car is the acceleration of the vehicle, β1 and β2 are the preset weight coefficients; c lane =-d center 2 , where d center is the distance between the vehicle and the center line of the road; c comfort =-(β3Δa car 2 +β4Δδ car 2 ), where Δa car and Δδ car are the vehicle's acceleration and steering angle change rate respectively, and β3 and β4 are preset weight coefficients.

[0021] Furthermore, the vehicle motion state, pedestrian motion state and lane line position state only retain the data within the preset range around the vehicle; and all lane line position states within the preset range around the vehicle are retained. For the pedestrian motion state within the preset range around the vehicle, the pedestrians are first sorted according to the distance from the pedestrian to the vehicle, and then the motion state of a preset number of pedestrians closest to the vehicle is retained, and if the number is insufficient, it is padded with 0.

[0022] Furthermore, the bottom lateral and longitudinal controller of the vehicle is a PID controller.

[0023] Furthermore, the neural network model parameters include a scene perception model, an interactively enhanced environment model, and a path planning model; wherein the scene perception model is used to extract the vehicle perception state from the vehicle sensor observation; the interactively enhanced environment model is used to model the interaction between vehicles, pedestrians, and the ground. Figure 3 The interactive relationship between the two is obtained, and the future vehicle perception state and objective function value are predicted; the path planning model is used to output the planning trajectory of the vehicle candidate; during training, the scene perception model and the interactive enhanced environment model are first trained in sequence using the driving data sequence in the experience replay pool, and then the path planning model is trained based on the behavior cloning method using the driving data sequence in the experience replay pool, so that the trained path planning model can generate the vehicle planning trajectory for a period of time in the future in an autoregressive manner; wherein, during the training process, the parameters of the scene perception model, the interactive enhanced environment model and the path planning model are updated using the gradient descent method.

[0024] Furthermore, the scene perception model includes a camera branch network and a radar branch network; wherein the camera branch network is composed of a 4-layer convolutional neural network and a 1-layer fully connected neural network, which is used to extract features from the image information collected by the vehicle surround view camera to obtain camera features. The radar branch network consists of a 4-layer convolutional neural network and a 1-layer fully connected neural network, which is used to extract features from the point cloud information collected by the vehicle laser radar to obtain radar features. For the obtained camera features and radar signature Use Transformer encoder and 1-layer fully connected neural network to extract multimodal fusion features Among them, the Transformer encoder is composed of a multi-layer self-attention mechanism, and its initial input is and The final output is the multimodal fusion feature Multimodal fusion features After passing through multiple decoder heads, the predicted value of the vehicle's motion state is output Prediction of pedestrian motion status And the predicted value of the lane line position state Among them, each decoder head consists of a 2-layer fully connected neural network.

[0025] Furthermore, the input of the interactively enhanced environment model is the vehicle motion state, the pedestrian motion state and the lane line position state; for the vehicle motion state, a two-layer fully connected neural network is first used for encoding, and then a self-attention mechanism is used to fuse the temporal features at the temporal level to obtain the vehicle motion state temporal feature f car; For the pedestrian motion state, a two-layer fully connected neural network is first used for encoding, and then a self-attention mechanism is used to fuse the temporal features at the temporal level to obtain the pedestrian motion state temporal features f obs ; For the lane line position state, a 2-layer fully connected neural network is used for encoding to obtain the lane line position state encoding information f line ;

[0026] In getting f car 、f obs and f line After that, the interactively enhanced environment model first uses the self-attention mechanism based on f car and f obs , modeling the interaction between cars and people, and then using the cross-attention mechanism to model the interaction between cars and maps, and between pedestrians and maps:

[0027]

[0028] Among them, I inter Represents the interaction result between the car and the person; I map Represents the interaction between the car and the map, and between pedestrians and the map; SA inter represents traffic participants, that is, the self-attention mechanism at the vehicle and pedestrian level, which is used to model the interaction between vehicles and pedestrians; Q, K, V represent the query, key value and value vector in the attention mechanism; Concat(f car ,f obs ) indicates f car and f obs The splicing result of CA line A cross-attention mechanism representing the traffic participant and road level is used to model the attention of traffic participants to lane line elements;

[0029] After obtaining the interaction results between the car and the person, the car and the map, and the pedestrian and the map, the interactively enhanced environment model generates three outputs according to the decoding of these interaction results. The first output is the prediction of the vehicle motion state and the pedestrian motion state at the next moment. By predicting the vehicle motion state and the pedestrian motion state at the next moment and supplementing the prediction results of the vehicle motion state and the pedestrian motion state to the existing vehicle operation state sequence and the pedestrian operation state sequence respectively, the interactively enhanced environment model can autoregressively generate future vehicle motion state sequences and pedestrian motion state sequences: wherein, the decoder for outputting the vehicle motion state prediction results and the decoder for outputting the pedestrian motion state prediction results are both composed of 2 layers of fully connected neural networks; the second output is the future trajectory prediction of pedestrians; wherein, the decoder for outputting the future trajectory prediction results of pedestrians is composed of 2 layers of fully connected neural networks; the third output is the objective function value prediction; wherein, the decoder for outputting the objective function value prediction results is composed of 2 layers of fully connected neural networks.

[0030] Furthermore, the input of the path planning model is f car 、f obs and f line ;

[0031] The path planning model first passes through f car Get a query vector and convert f obs As the key vector and value vector, the cross attention mechanism is used to obtain the interaction features between vehicles and pedestrians; the interaction features between vehicles and pedestrians are then used as the query vector, and f line As key-value vectors and value vectors, the cross-attention mechanism is used to obtain the interaction features between the vehicle and the lane boundary; the interaction features between the vehicle and the pedestrian and the interaction features between the vehicle and the lane boundary are fused and handed over to the planning decoder to output the final motion planning trajectory; wherein the planning decoder uses a gated recurrent unit to generate future planned trajectory points in an autoregressive manner.

[0032] Furthermore, the path planning model first generates multiple planning trajectories based on the multiple targeting method, and then uses the interactively enhanced environment model to simulate the multiple planning trajectories generated by the path planning model, and estimates the sum of the objective function values ​​of each trajectory at each time step, and selects the trajectory with the largest sum of the objective function values ​​at each time step; the selected trajectory is used as the actual running trajectory, and the vehicle's underlying lateral and longitudinal controller is used in the Carla simulator to control the vehicle to complete the trajectory.

[0033] On the other hand, the present invention further provides an electronic device, comprising a processor and a memory; wherein the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the above method.

[0034] In yet another aspect, the present invention further provides a computer-readable storage medium, wherein at least one instruction is stored in the storage medium, and the instruction is loaded and executed by a processor to implement the above method.

[0035] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0036] First, the environmental model obtained through learning in the present invention can adapt to system changes and nonlinearity, and the interaction enhancement technology further establishes the interactive relationship between the car, pedestrians and maps, and models the driving scene more accurately; secondly, by imitating and learning the trajectory of human experts, the path planning model in the present invention can provide an anthropomorphic high-quality trajectory initial solution, which helps to improve the efficiency and final quality of planning, especially in complex driving environments such as pedestrian interaction. Finally, the present invention can realize dynamic path planning for autonomous driving cars in pedestrian interaction scenarios. This method can not only respond to changes in pedestrian behavior in real time, but also optimize the efficiency and comfort of intelligent driving in pedestrian interaction scenarios while ensuring safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0038] Figure 1 is a flow chart of an intelligent driving planning method for pedestrian interaction scenarios provided by an embodiment of the present invention;

[0039] Figure 2 Schematic diagram of a training framework of an intelligent driving planning method for pedestrian interaction scenarios provided by an embodiment of the present invention;

[0040] Figure 3 It is a schematic diagram of a simulation of a pedestrian interactive driving scenario in a park environment provided by an embodiment of the present invention;

[0041] Figure 4 is a schematic diagram of a neural network of an interactively enhanced environment model provided by an embodiment of the present invention;

[0042] Figure 5 is a schematic diagram of a trajectory planning model neural network provided by an embodiment of the present invention;

[0043] Figure 6 It is a system block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0045] First of all, it should be noted that in the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "exemplarily" is intended to present the concept in a concrete way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0046] First embodiment

[0047] In view of the problems of poor trajectory quality, low interactivity and lack of anthropomorphism in existing intelligent driving planning methods in complex interactive scenarios, this embodiment follows the idea of ​​MPC, introduces the learned environment model and path planning model, and implements an intelligent driving planning method for pedestrian interaction scenarios. Its system architecture includes: a campus pedestrian interaction scenario built based on the Carla simulator, which is used to support the training and verification of the algorithm, define the input and objective function required for the algorithm to run, and the underlying controller of the vehicle; an experience replay pool, which is used to store the experience collected by experts in the simulation scenario as training data for the algorithm; a parameter initializer, which is used to initialize the neural network model parameters of the algorithm; a scene perception model, which is used to process the sensor observations of the car and extract the features of the car itself and the surrounding environment (including road boundaries and pedestrians); an interactively enhanced environment model, which is used to model the interactive relationships between pedestrians, between pedestrians and cars, and between traffic participants including cars and pedestrians and road boundaries, and predict changes in the environment around the car and target values ​​based on these interactive relationships; a path planning model, which is used to output candidate planning paths in the next few seconds based on the features obtained from the upstream model. During the operation of the algorithm, the parameter initializer is first used to initialize the network parameters of the scene perception model, the interactively enhanced environmental model, and the path planning model, and the experience replay pool is initialized. Then the following steps are repeated until the algorithm performance reaches the expected level: human experts drive the car in the park simulation scene, complete the car's navigation tasks under various pedestrian densities and road conditions, collect relevant driving data, and store it in the experience replay pool; use the data in the experience pool to perform multiple rounds of updates on the scene perception model, the interactively enhanced environmental model, and the path planning model; use the above three models in the simulation environment to complete the intelligent driving planning task in the pedestrian interaction scene, check the algorithm performance, and human experts collect data again for driving scenarios that do not meet the performance standards, replenish the experience replay pool, and conduct the next training until the performance in all scenarios meets the standards.

[0048] Specifically, the execution process of this method is as follows Figure 1 As shown, the following steps are included:

[0049] S1, build a pedestrian interactive driving scenario in a campus environment, and define vehicle sensor observation, vehicle perception status, objective function, and vehicle underlying lateral and longitudinal controllers;

[0050] Among them, this embodiment builds a pedestrian interactive driving scene in a campus environment based on the Carla simulator, and creates a set of pedestrian generation and interaction mechanisms, which effectively improves the authenticity of the simulation environment. Specifically, this embodiment uses Roadrunner to create a map in the OpenDRIVE format based on the actual road structure of a certain campus. In order to simulate the actual pedestrian interaction scene, 100 pedestrians with different shapes and behaviors are generated in the map, and these pedestrians are randomly assigned starting positions and target positions so that they walk along possible paths. When the pedestrians reach the predetermined target position, they will be assigned a new target position so that the generated pedestrians move in the campus. By introducing dynamic traffic flows and uncertain pedestrian behavior patterns, the simulation scene can simulate different interaction conditions, thereby enhancing the authenticity and complexity of the scene. Its visual interface is as follows: Figure 3 shown.

[0051] In addition, it should be noted that the road boundary position features around the car and the motion features of the car and pedestrians need to be further processed before they can be input into the downstream environmental model and the planning path planning model. Among them, for all features, only those within 40m around the car are retained first; for the road position features, all are retained, and for the motion state features of pedestrians, they are first sorted by the distance to the car, and then the 20 most recent pedestrian motion features are retained, and if they are insufficient, they are padded with 0.

[0052] Furthermore, the vehicle sensor observation o includes the following information: vehicle surround camera o cam and vehicle lidar lidar ; Among them, the surround camera input dimension is Among them, the number of surround view cameras is 6, the number of image channels (RGB) of each camera is 3, and the resolution of a single camera image is 1280x720. High-resolution input can provide rich visual information, which is conducive to accurate environmental perception and object recognition; the lidar point cloud is converted into a bird's-eye view with a fixed resolution. Considering the points 20 meters in front and behind and 20 meters to the left and right of the vehicle, a square grid with a side length of 40 meters is formed. The grid is divided into blocks of 0.125 meters by 0.125 meters. The resolution of the bird's-eye view is 320x320. Further considering the quantization of the height information of each grid block into 4 possible levels, the final lidar input dimension is Such processing enables the radar bird's-eye view to capture the relative height changes of the ground and different obstacles, enhance the understanding of the three-dimensional structure of the surrounding environment, and provide the vehicle with more accurate spatial information.

[0053] Furthermore, the vehicle perception state s includes the following information: vehicle motion state s car , pedestrian motion state s obs , lane line position state s line The dimension of the car's motion state is Contains the global coordinates of the car (x car ,y car ), speed v car , acceleration a car , towards θ car , and the width and length of the vehicle body (width, length); the pedestrian motion state dimension is Where N obs is the number of pedestrians, generally set to 20. The motion state of a single pedestrian includes the global coordinates of the pedestrian (x obs ,y obs ), speed v obs , acceleration a obs , and the direction θ obs ; The lane line position state dimension is Among them, N line is the number of lane lines, which is generally set to 200. A single lane line contains 20 2D coordinate points. For pedestrian status, if there are more than 20 pedestrians within the perception range, the nearest 20 are selected according to the distance to the car. If the number is insufficient, it is padded with 0; for lane line status, if the number is insufficient, it is also padded with 0. The above status information can be directly obtained through the simulator to obtain the true value when collecting human expert data, but it is forbidden to use the simulator to obtain the true value when testing the trained model. The predicted value of this information can only be obtained through the environmental perception model.

[0054] Furthermore, the vehicle objective function c is calculated based on the vehicle perception state and control action, mainly including the safety objective c safe , efficiency target c speed , lane keeping target c lane And comfort target c comfort Specifically, the security goal c safe The purpose is to make the car avoid collision with pedestrians and keep as far away from pedestrians as possible, defined as Among them, N obs =20 is the number of pedestrians, d safe =1m is the preset safety distance, d i is the distance between the ith pedestrian and the car; the efficiency target c speedIt aims to optimize the driving path and speed of the car to reduce the driving time, which is defined as c speed =β1v car +β2a car , where v car is the speed of the car, a car is the acceleration of the car, β1 and β2 are weight coefficients, usually set to 1 and 0.2 respectively; the lane keeping target c lane It aims to ensure that the car drives stably in the lane and avoids deviating from the center of the lane. This is achieved by minimizing the deviation between the car and the center of the lane, which is defined as c lane =-d center 2 , where d center The distance between the car and the center line of the road; comfort target c comfort It aims to reduce the discomfort of the car passengers by smoothing the acceleration and steering angle changes of the car, which is defined as c comfort =-(β3Δa car 2 +β4Δδ car 2 ), where Δa car and Δδ car are the acceleration and steering angle change rate of the car, β3 and β4 are weight coefficients, usually set to 0.1 and 0.1. In general, the objective function c is the weighted sum of the four sub-objective functions: c = α1c safe +α2c speed +α3c lane +α4c comfort , where α1~α4 are the weighting coefficients of each item, usually set to 1, 0.1, 0.5, 0.1.

[0055] Furthermore, the vehicle's underlying lateral and longitudinal controller adopts a proportional-integral-derivative (PID) controller, and the vehicle's underlying lateral and longitudinal controller can generate two control actions, acceleration a and steering angle δ, for tracking the future trajectory of the car output by the path planning model.

[0056] S2, initialize the neural network model parameters and initialize the experience replay pool;

[0057] The neural network model is used to realize the mapping and pedestrian detection tasks, predict the state transition of the environment, and generate candidate planning trajectories; the experience replay pool is used to store the driving data sequence of human experts, where the capacity of the experience replay pool is limited and the storage rule is the first-in-first-out rule; a single expert data point at time t can be expressed as (o t ,s t ,c t), each data sequence can be represented by Indicates, H indicates the length of the data sequence, o t is the sensor observation of the car at time t, including the surround camera and lidar; s t is the state of the car at time t, including the position, direction, speed and other motion state characteristics of the car and pedestrians, as well as the position characteristics of the lane line; c t It is the objective function of the car at time t, indicating the benefit of the car’s driving at this time.

[0058] Specifically, in this embodiment, the neural network model parameters include three parts: scene perception model, interactive enhanced environment model and path planning model; the scene perception model is used to extract the vehicle perception state from the vehicle sensor observation; the interactive enhanced environment model is used to model the interaction between vehicles, pedestrians and the ground. Figure 3 The interactive relationship between the two is used to predict the future vehicle perception state and the objective function value; the path planning model is used to output the planning trajectory of the vehicle candidate; the interactively enhanced environment model and path planning model are composed of a fully connected neural network; below, combined with Figure 2 , Figure 4 and Figure 5 Each model is introduced in detail:

[0059] The scene perception model is used to obtain the driving scene status around the car from the surround view camera and the on-board laser radar. Figure 2 As shown in the figure, it uses two branches to extract features from camera input and radar input respectively:

[0060]

[0061] Among them, Extracter cam and Extracter lidar Represents the camera branch network and the radar branch network respectively. cam It consists of 4 layers of convolutional neural networks and 1 layer of fully connected neural networks, which extract the visual features of 6 surround cameras respectively. Same Extracter lidar It also consists of 4 layers of convolutional neural network and 1 layer of fully connected neural network.

[0062] get and Finally, the Transformer encoder and a 1-layer fully connected neural network (MLP) are used to extract multimodal fusion features. Among them, Transformer consists of a multi-layer self-attention mechanism, and its initial input is and Concat, and finally output multimodal fusion features

[0063]

[0064] Multimodal fusion features After passing through multiple decoder heads, the predicted values ​​of the vehicle motion state, pedestrian motion state and lane line position state are output

[0065]

[0066] Among them, Decoder car ,Decoder obs ,Decoder line They are decoders for the car, pedestrian and lane status, respectively, and in this embodiment, they are all composed of a 2-layer fully connected neural network.

[0067] Furthermore, the interactively enhanced environment model first uses encoders to further encode the lane position feature sequences and the motion feature sequences of cars and pedestrians extracted by the perception module; secondly, multiple self-attention mechanisms and cross-attention mechanisms are introduced for the extracted features to model temporal features and spatial interaction features; after integrating the above features, a prediction task is performed for all features at the next moment, a trajectory prediction task is performed for pedestrian-related features, and a regression prediction task is performed for each sub-goal in the objective function to achieve model training. Specifically, the structure of the interactively enhanced environment model is as follows: Figure 4 As shown, its input is the motion state sequence of cars and pedestrians in the expert data: Among them, H is the sequence length, and the lane line position status The state sequences of cars and pedestrians and the state of lane lines are encoded using a two-layer fully connected neural network (Multilayer Perceptron, MLP). For the encoded results of cars and pedestrians, a self-attention mechanism is used to fuse the temporal features at the temporal level:

[0068]

[0069] In the above formula, f car , f car , f car Represent the encoding results of cars, pedestrians and lane lines respectively, SA time represents the self-attention mechanism at the time level, Q, K, V represent the query, key, and value vectors in the attention mechanism.

[0070] After obtaining the encoding results, the environment model first uses the self-attention mechanism to model the interaction between the car and the human, and then uses the cross-attention mechanism to model the interaction between the car and the map, and between the pedestrian and the map:

[0071]

[0072] In the above formula, I inter Represents the interaction result between the car and the person, I map Represents the interaction between the car and the map, and between pedestrians and the map. inter Represents traffic participants, that is, the self-attention mechanism at the car and pedestrian level, which is used to model the interaction between cars and pedestrians; CA line A cross-attention mechanism representing the traffic participant and road levels is used to model the attention of traffic participants to lane line elements.

[0073] After obtaining the interaction results, the environment model decodes these interaction results to generate three outputs. The first output is the prediction of the motion features at the next moment. By predicting the features at the next moment and adding them to the existing state sequence, the environment model can autoregressively generate the future state sequence:

[0074]

[0075] In the above formula, Decoder pred_car and Decoder pred_obs They are predictors of car and pedestrian features, respectively, and in this embodiment, they are composed of 2 layers of fully connected neural networks.

[0076] The second output is the pedestrian’s future trajectory prediction. By introducing the pedestrian’s future trajectory prediction task, the network supervision signal is enhanced and the model has a certain degree of post-interpretability:

[0077]

[0078] In the above formula Represents the pedestrian's future trajectory prediction, and its true value Extract from the dataset; Decoder pred_traj is a trajectory predictor, which in this embodiment consists of a 2-layer fully connected neural network.

[0079] The third output is the prediction of the objective function value. Considering that the objective function contains 4 sub-objectives, the model also predicts the values ​​of the four sub-objective functions separately:

[0080]

[0081] In the above formula They are the predictions of safety, driving efficiency, lane keeping and comfort goals; Decoder cost_safe , Decoder cost_speed , Decoder cost_lane , Decoder cost_comfort They are respectively predictors of each sub-goal, and in this embodiment, they are composed of a 2-layer fully connected neural network.

[0082] The interactively enhanced environmental model can take into account the interactions between various elements in the driving scene, and can better establish the dynamic relationship between vehicles, pedestrians, and road environments, so that the autonomous driving system can more accurately predict and respond to changes in the surrounding environment, thereby simulating and evaluating complex traffic scenarios.

[0083] Furthermore, the input of the path planning model is the time sequence code f of the car. car , the temporal encoding of pedestrians f obs , and the road location code f line . The path planning model first obtains a query (Query) vector through the current features of the car, takes the pedestrian features as the key (Key) vector and the value (Value) vector, and uses the cross-attention mechanism to obtain the interaction features between the car and the pedestrian; then the interaction features obtained in the previous step are used as the query (Query) vector, and the lane line features are used as the key (Key) vector and the value (Value) vector, and the cross-attention mechanism is used to obtain the interaction features between the car and the lane boundary; the above two interaction features are combined and handed over to the planning decoder to output the final motion planning trajectory. The planning decoder uses a gated recurrent unit to generate future planned trajectory points in an autoregressive manner. Specifically, the structure of the path planning model is as follows Figure 5 shown.

[0084] It should be noted that the planning of the car also needs to consider the interaction with pedestrians and map information, which is achieved using the cross-attention mechanism in this embodiment:

[0085]

[0086] In the above formula, I car-obs It is the interaction between the car and the pedestrian, representing the car's attention to the pedestrian; I car-line It is the interaction between the car and the road, representing the car's attention to the lane line. car-obs and CA car-line They are the cross-attention mechanisms between cars and pedestrians and between cars and roads.

[0087] After obtaining the above two interaction results, they are input to the planning decoder. In this example, the planning decoder is composed of a gated recurrent unit (GRU). The working mode of GRU at time t can be expressed as:

[0088]

[0089] In the above formula, h t represents the latent state at time t, and the initial time, i.e., the latent state at time 0, is concatenated by I car-obs with I car-line The latent state is generated by GRU at the subsequent moment; f t+1 It is the intermediate feature of planning at time t+1. Through this feature, Gaussian distribution can be constructed and sampled to obtain the offset Δw of the coordinate value of the planned trajectory point at time t+1 t+1 The final predicted planning trajectory It can be expressed as:

[0090]

[0091] In the above formula, T is the step length of the planned trajectory of the car, and the true value p of the planned trajectory is car Extracted from the dataset.

[0092] S3, human experts drive vehicles in the pedestrian interaction driving scenario in the campus environment, complete vehicle navigation tasks under various pedestrian densities and various road conditions, generate corresponding driving data sequences, and store them in the experience replay pool;

[0093] S4, training the neural network model using the driving data sequence in the experience replay pool until a preset training termination condition is reached;

[0094] Specifically, in this embodiment, the training process of the model is: sampling expert data sequence samples from the experience playback pool, training the scene perception model and the interactive enhanced environment model, so that the scene perception model can obtain the state characteristics of the car, pedestrians and lane lines from the sensor observation, so that the environment model can model the vehicle-human interaction environment, predict state changes, and estimate the value of the objective function. After the scene perception model is trained and the state information can be accurately obtained from the sensor input, the interactive enhanced environment model is trained, specifically: using the expert data set, the path planning model is trained based on the behavior cloning method, so that the path planning model can generate the planned trajectory of the car for a period of time in the future in an autoregressive manner.

[0095] During the training process, the gradient descent method is used to update the parameters of the scene perception model, the interactively enhanced environment model, and the path planning model, and it is determined in real time whether the preset training termination condition is met, that is, whether the number of training iteration rounds reaches the upper limit. If not, the training continues until the preset training termination condition is met; the number of training iteration rounds is also usually referred to as epoch. In this embodiment, it is set to 60.

[0096] S5, deploys the trained neural network model in the pedestrian interactive driving scenario in the campus environment, plans the vehicle's running trajectory, and uses the vehicle's underlying lateral and longitudinal controllers to control the vehicle to run according to the planned trajectory;

[0097] Furthermore, the path planning model first generates multiple planned trajectories based on the multiple targeting method, and then uses the interactive enhanced environment model to simulate the multiple trajectories, and estimates the sum of the objective function values ​​of each trajectory at each time step, and selects the trajectory with the largest sum of the objective function values ​​at each time step; the selected trajectory is used as the actual running trajectory, and the underlying lateral and longitudinal PID controller is used in the Carla simulator to control the car to complete the trajectory.

[0098] Specifically, since the coordinate value offset of the planned trajectory point is sampled from the distribution, multiple candidate planning trajectories can be obtained by sampling the planning model multiple times according to the multiple shooting method. In this embodiment, the number of sampled candidate planning trajectories is set to 5. When the future trajectory of the car is determined, the future motion state characteristics of the car are also determined. Based on the fixed historical motion characteristics of the car and pedestrians, as well as the future motion characteristics of the car, the environmental model can predict the motion characteristics of the future pedestrians in the form of autoregression and estimate the future objective function value. By calculating the objective function values ​​of the five candidate trajectories, the trajectory with the largest sum is selected as the actual running trajectory.

[0099] S6, judging whether the performance of the neural network model meets the standard. If the performance of the neural network model does not meet the standard in a certain scenario, more driving data sequences in the scenario that does not meet the standard are collected by human experts, stored in the experience replay pool, and S4 to S5 are repeated until the performance of the neural network model meets the standard in all scenarios;

[0100] S7, the training is finished, and the last updated neural network model is obtained as the training result.

[0101] The final trained model can be used to realize intelligent driving planning for pedestrian interaction scenarios.

[0102] In summary, this embodiment provides an intelligent driving planning method for pedestrian interaction scenarios. First, the environmental model obtained through learning can adapt to system changes and nonlinearity, and the interaction enhancement technology further establishes the interactive relationship between the car, pedestrians and maps, and models the driving scene more accurately; secondly, by imitating and learning the trajectory of human experts, the path planning model can provide an anthropomorphic high-quality trajectory initial solution, which helps to improve the efficiency and final quality of planning, especially in complex driving environments such as pedestrian interaction. Finally, the intelligent driving planning method of this embodiment can realize dynamic path planning for autonomous driving cars in pedestrian interaction scenarios. This method can not only respond to changes in pedestrian behavior in real time, but also optimize the efficiency and comfort of intelligent driving in pedestrian interaction scenarios while ensuring safety.

[0103] Second embodiment

[0104] This embodiment provides an electronic device, such as Figure 6 As shown, the electronic device includes: a processor and a memory; wherein the processor and the memory can be connected via a communication bus; the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the method of the first embodiment. In addition, the electronic device may also include a transceiver, the processor and the transceiver can be connected via a communication bus, and the transceiver is used to communicate with other devices.

[0105] Next, combine Figure 6 The following is a detailed introduction to the various components of the electronic device:

[0106] Among them, the processor is the control center of the electronic device, and the electronic device may include multiple processors, each of which may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may be a processor or a general term for multiple processing elements. For example, the processor is one or more central processing units (CPUs), or other general-purpose processors, application specific integrated circuits (ASICs), or one or more integrated circuits configured to implement an embodiment of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (field programmable gate arrays, FPGAs), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor may execute various functions of the electronic device by running or executing software programs stored in the memory and calling data stored in the memory.

[0107] In a specific implementation, as an embodiment, the processor may include one or more CPUs, such as Figure 6 The CPU0 and CPU1 shown in the figure are, of course, only exemplary.

[0108] The memory is used to store the software program for executing the solution of the present invention, and the execution is controlled by the processor. The specific implementation method can refer to the above method embodiment and will not be repeated here.

[0109] Optionally, the memory may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may be integrated with the processor or exist independently and accessed through the interface circuit ( Figure 6 (not shown) is coupled to the processor, which is not specifically limited in this embodiment of the present invention.

[0110] The transceiver may include a receiver and a transmitter ( Figure 6 The receiver is used to implement the receiving function, and the transmitter is used to implement the sending function. The transceiver can be integrated with the processor or exist independently and communicate with the electronic device through the interface circuit ( Figure 6 (not shown) is coupled to the processor, which is not specifically limited in this embodiment of the present invention.

[0111] In addition, it should be noted that Figure 6 The structure of the electronic device shown in the figure does not constitute a limitation on the device, and the actual device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. In addition, the technical effects achieved by the electronic device when executing the method of the first embodiment above can refer to the technical effects described in the first embodiment above, so they are not repeated here.

[0112] Third embodiment

[0113] This embodiment provides a computer-readable storage medium, which stores at least one instruction, and the instruction is loaded and executed by a processor to implement the method of the first embodiment. The computer-readable storage medium may be a ROM, a random access memory, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc. The instructions stored therein may be loaded by a processor in a terminal to execute the method.

[0114] In addition, it should be noted that the present invention can be provided as a method, an apparatus or a computer program product. Therefore, the embodiment of the present invention can be in the form of a full or partial hardware embodiment, a full or partial software embodiment or an embodiment combining software and hardware. Moreover, when implemented using software, the embodiment of the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program codes. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center containing one or more available media sets. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium. The semiconductor medium may be a solid state hard disk.

[0115] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0116] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0117] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements. In addition, the term "and / or" is only an association relationship describing the associated objects, indicating that there can be three relationships, for example, A and / or B, which can represent: A exists alone, A and B exist at the same time, and B exists alone, wherein A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding. "At least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can be represented by: a, b, c, ab, ac, bc or abc, where a, b, c can be single or plural.

[0118] In addition, it can be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0119] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0120] In several embodiments provided by the present invention, it should be understood that the disclosed equipment, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of functional modules / units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0121] If the method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0122] Finally, it should be noted that the above is only a preferred embodiment of the present invention. It should be pointed out that although the preferred embodiment of the present invention has been described, for ordinary technicians in this technical field, once the basic creative concept of the present invention is known, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the protection scope of the present invention. Therefore, the attached claims are intended to be interpreted as including the preferred embodiment and all changes and modifications that fall within the scope of the embodiments of the present invention.

Claims

1. An intelligent driving planning method for pedestrian interaction scenarios, characterized in that: include: S1, build a pedestrian interactive driving scenario in a campus environment, and define vehicle sensor observation, vehicle perception status, objective function, and vehicle underlying lateral and longitudinal controllers; S2, initialize the neural network model parameters and the experience replay pool; the neural network model is used to generate candidate planning trajectories; the experience replay pool is used to store driving data sequences of human experts, each of which consists of vehicle sensor observations, vehicle perception states, and objective function values ​​at multiple moments; S3, human experts drive vehicles in the pedestrian interaction driving scenario in the campus environment, complete vehicle navigation tasks under various pedestrian densities and various road conditions, generate corresponding driving data sequences, and store them in the experience replay pool; S4, training the neural network model using the driving data sequence in the experience replay pool until a preset training termination condition is reached; S5, deploys the trained neural network model in the pedestrian interactive driving scenario in the campus environment, plans the vehicle's running trajectory, and uses the vehicle's underlying lateral and longitudinal controllers to control the vehicle to run according to the planned trajectory; S6, judging whether the performance of the neural network model meets the standard. If the performance of the neural network model does not meet the standard in a certain scenario, more driving data sequences in the scenario that does not meet the standard are collected by human experts, stored in the experience replay pool, and S4 to S5 are repeated until the performance of the neural network model meets the standard in all scenarios; S7, the training is finished, and the last updated neural network model is obtained as the training result.

2. The intelligent driving planning method for pedestrian interaction scenarios according to claim 1, characterized in that: The construction of a pedestrian interactive driving scene in a park environment includes: Based on the Carla simulator, a pedestrian interactive driving scenario in a campus environment is built. According to the actual campus scenario, the corresponding OpenDRIVE format map is constructed, and multiple pedestrians with different shapes and behaviors are randomly generated. The starting position and target position are randomly assigned to these pedestrians so that they walk along possible paths. When the pedestrians reach the predetermined target position, a new target position is assigned to them, so that the generated pedestrians move in the campus.

3. The intelligent driving planning method for pedestrian interaction scenarios according to claim 1, characterized in that: The vehicle sensor observation includes image information collected by the vehicle surround view camera and point cloud information collected by the vehicle laser radar; wherein the point cloud information collected by the vehicle laser radar is converted into a bird's-eye view with a fixed resolution, forming a square grid centered on the vehicle with a preset length as the side length, dividing the grid into a plurality of grid blocks of the same size, and quantizing the height information of each grid block into a preset number of levels; The vehicle perception state includes the vehicle motion state, the pedestrian motion state and the lane line position state; wherein the vehicle motion state includes the vehicle's global coordinates, speed, acceleration, direction and the width and length of the vehicle body; the pedestrian motion state includes the pedestrian's global coordinates, speed, acceleration and direction; the lane line position state includes multiple two-dimensional coordinate points of each lane line; The objective function is expressed as: c=α1c safe +α2c speed +α3c lane +α4c comfort Wherein, c represents the objective function; α1, α2, α3, α4 are preset weighting coefficients; Among them, N obs is the number of pedestrians, d safe is the preset safety distance, d i is the distance between the ith pedestrian and the vehicle; c speed =β1v car +β2a car , where v car is the speed of the vehicle, a car is the acceleration of the vehicle, β1 and β2 are the preset weight coefficients; c lane =-d center 2 , where d center is the distance between the vehicle and the center line of the road; c comfort =-(β3Δa car 2 +β4Δδ car 2 ), where Δa car and Δδ car are the vehicle's acceleration and steering angle change rate respectively, and β3 and β4 are preset weight coefficients.

4. The intelligent driving planning method for pedestrian interaction scenarios according to claim 3, characterized in that: The vehicle motion state, pedestrian motion state and lane line position state only retain the data within the preset range around the vehicle; and all lane line position states within the preset range around the vehicle are retained. For the pedestrian motion state within the preset range around the vehicle, the pedestrians are first sorted according to the distance from the pedestrian to the vehicle, and then the motion states of a preset number of pedestrians closest to the vehicle are retained, and if the number is insufficient, it is padded with 0.

5. The intelligent driving planning method for pedestrian interaction scenarios according to claim 1, characterized in that: The vehicle bottom layer transverse and longitudinal controller is a PID controller.

6. The intelligent driving planning method for pedestrian interaction scenarios according to claim 3, characterized in that: The neural network model parameters include a scene perception model, an interactively enhanced environment model, and a path planning model; wherein the scene perception model is used to extract the vehicle perception state from the vehicle sensor observation; the interactively enhanced environment model is used to model the interaction relationship between the vehicle, pedestrians, and the map, and predict the future vehicle perception state and objective function value; the path planning model is used to output the planning trajectory of the vehicle candidate; During training, the scene perception model and the interactively enhanced environment model are first trained in sequence using the driving data sequence in the experience replay pool, and then the path planning model is trained based on the behavior cloning method using the driving data sequence in the experience replay pool, so that the trained path planning model can generate the vehicle planning trajectory for a period of time in the future in an autoregressive manner; wherein, during the training process, the parameters of the scene perception model, the interactively enhanced environment model and the path planning model are updated using the gradient descent method.

7. The intelligent driving planning method for pedestrian interaction scenarios according to claim 6, characterized in that: The scene perception model includes a camera branch network and a radar branch network; wherein the camera branch network is composed of a 4-layer convolutional neural network and a 1-layer fully connected neural network, which is used to extract features from the image information collected by the vehicle surround view camera to obtain camera features. The radar branch network consists of a 4-layer convolutional neural network and a 1-layer fully connected neural network, which is used to extract features from the point cloud information collected by the vehicle laser radar to obtain radar features. For the obtained camera features and radar signature Use Transformer encoder and 1-layer fully connected neural network to extract multimodal fusion features The Transformer encoder is composed of a multi-layer self-attention mechanism, and its initial input is and The final output is the multimodal fusion feature Multimodal fusion features After passing through multiple decoder heads, the predicted value of the vehicle's motion state is output Prediction of pedestrian motion status And the predicted value of the lane line position state Among them, each decoder head consists of a 2-layer fully connected neural network.

8. The intelligent driving planning method for pedestrian interaction scenarios according to claim 7, characterized in that: The input of the interactive enhanced environment model is the vehicle motion state, pedestrian motion state and lane line position state. For the vehicle motion state, a two-layer fully connected neural network is first used for encoding, and then a self-attention mechanism is used to fuse the temporal features at the temporal level to obtain the vehicle motion state temporal feature f car ; For the pedestrian motion state, a two-layer fully connected neural network is first used for encoding, and then a self-attention mechanism is used to fuse the temporal features at the temporal level to obtain the pedestrian motion state temporal features f obs ; For the lane line position state, a 2-layer fully connected neural network is used for encoding to obtain the lane line position state encoding information f line ; In getting f car 、f obs and f line After that, the interactively enhanced environment model first uses the self-attention mechanism based on f car and f obs , modeling the interaction between cars and people, and then using the cross-attention mechanism to model the interaction between cars and maps, and between pedestrians and maps: Among them, I inter Represents the interaction result between the car and the person; I map Represents the interaction between the car and the map, and between pedestrians and the map; SA inter represents traffic participants, that is, the self-attention mechanism at the vehicle and pedestrian level, which is used to model the interaction between vehicles and pedestrians; Q, K, V represent the query, key value and value vector in the attention mechanism; Concat(f car ,f obs ) indicates f car and f obs The splicing result of CA line A cross-attention mechanism representing the traffic participants and the road level is used to model the attention of traffic participants to lane line elements; After obtaining the interaction results between the car and the person, the car and the map, and the pedestrian and the map, the interactively enhanced environment model generates three outputs based on the decoding of these interaction results. The first output is the prediction of the vehicle motion state and the pedestrian motion state at the next moment. By predicting the vehicle motion state and the pedestrian motion state at the next moment and adding the predicted result of the vehicle motion state to the existing vehicle running state sequence, and adding the predicted result of the pedestrian motion state to the existing pedestrian running state sequence, the interactively enhanced environment model can autoregressively generate future vehicle motion state sequences and pedestrian motion state sequences: wherein the decoder for outputting the vehicle motion state prediction result and the decoder for outputting the pedestrian motion state prediction result are respectively composed of 2 layers of fully connected neural networks; the second output is the future trajectory prediction of pedestrians; wherein the decoder for outputting the future trajectory prediction result of pedestrians is composed of 2 layers of fully connected neural networks; the third output is the objective function value prediction; wherein the decoder for outputting the objective function value prediction result is composed of 2 layers of fully connected neural networks.

9. The intelligent driving planning method for pedestrian interaction scenarios according to claim 8, characterized in that: The input of the path planning model is f car 、f obs and f line ; The path planning model first passes through f car Get a query vector and convert f obs As the key vector and value vector, the cross attention mechanism is used to obtain the interaction features between vehicles and pedestrians; the interaction features between vehicles and pedestrians are then used as the query vector, and f line As key-value vectors and value vectors, the cross-attention mechanism is used to obtain the interaction features between the vehicle and the lane boundary; the interaction features between the vehicle and the pedestrian and the interaction features between the vehicle and the lane boundary are fused and handed over to the planning decoder to output the final motion planning trajectory; wherein the planning decoder uses a gated recurrent unit to generate future planned trajectory points in an autoregressive manner.

10. The intelligent driving planning method for pedestrian interaction scenarios according to claim 9, characterized in that: The path planning model first generates multiple planning trajectories based on the multiple targeting method, and then uses the interactive enhanced environment model to simulate the multiple planning trajectories generated by the path planning model, and estimates the sum of the objective function values ​​of each trajectory at each time step, and selects the trajectory with the largest sum of the objective function values ​​at each time step; the selected trajectory is used as the actual running trajectory, and the vehicle's underlying lateral and longitudinal controller is used in the Carla simulator to control the vehicle to complete the trajectory.

Images