A method for autonomous vehicle traffic decision-making in urban areas with no traffic signals

Abstract

This invention relates to the field of traffic safety technology, and more particularly to a method for autonomous vehicle traffic decision-making in urban unsignalized road sections. The method includes: Step S1, collecting motion state information of the autonomous vehicle and acquiring pedestrian images and motion state information through onboard environmental perception sensors; Step S2, estimating the coordinates of pedestrian skeletal key points using the Alphapose algorithm and converting the key points into a feature matrix; Step S3, using a GRU-Transformer network to predict pedestrian crossing intentions at crosswalks; Step S4, modeling the autonomous vehicle traffic decision at crosswalks as a Markov decision process; Step S5, constructing an autonomous vehicle traffic strategy based on an inverse reinforcement learning model and fitting a reward function using a deep neural network; and Step S6, training the inverse reinforcement learning model. This invention can improve the safety, efficiency, and comfort of autonomous vehicle traffic at crosswalks.

Description

Technical Field

[0001] This invention relates to the field of traffic safety technology, and in particular to a method for making traffic decisions for autonomous vehicles on urban unsignaled road sections. Background Technology

[0002] Pedestrian crossings often lack traffic signal guidance and fail to separate pedestrian and autonomous vehicle right-of-way in terms of time, easily leading to conflicts between pedestrians and vehicles. Currently, when autonomous vehicles interact with pedestrians, they typically adopt a conservative strategy of unconditionally yielding to pedestrians to ensure driving safety, resulting in low traffic efficiency and a potential for rear-end collisions. Therefore, it is crucial to research a decision-making method for autonomous vehicle traffic at pedestrian crossings that combines high safety, high efficiency, and comfort.

[0003] Existing research on autonomous vehicle traffic decision-making at pedestrian crossings is mostly based on empirical or semi-empirical methods. It relies solely on human experience to construct rules, making it difficult to reveal the implicit information within the driver's decision-making behavior. The scenario coverage is limited and does not involve complex scenarios, thus failing to achieve truly driver-like decision-making.

[0004] Chinese Patent Publication No. CN111391830B discloses a longitudinal decision-making system for autonomous vehicles based on distance parameter judgment, comprising: a distance calculation unit for calculating distance parameters of the vehicle relative to a vehicle ahead in the same lane, wherein the distance parameters include one or more of the following distance, AEB distance, expected following distance, and maximum speed limit distance; a state switching unit for switching longitudinal driving states according to the distance parameters and the vehicle's environmental information; and a state determination unit for determining and outputting the current longitudinal driving state of the vehicle based on the switching result of the state switching unit. It is evident that the prior art has the following problems: the prior art mainly sets collision avoidance rules based on real-time distance parameter changes, without considering the need to correct the traffic strategy according to the actual driving strategy of the autonomous vehicle, thus affecting the driving safety, efficiency, and comfort of autonomous vehicles in dynamic and complex human-vehicle interaction scenarios. Summary of the Invention

[0005] Therefore, this invention provides a method for autonomous vehicle traffic decision-making in urban unsignaled road sections, which overcomes the problem in the prior art that does not consider the correction of traffic strategy based on the actual driving strategy of the autonomous vehicle, thus affecting the driving safety, efficiency and comfort of autonomous vehicles in dynamic and complex human-vehicle interaction scenarios.

[0006] To achieve the above objectives, the present invention provides a method for autonomous vehicle traffic decision-making in urban areas with no traffic signals, comprising:

[0007] Step S1: When the autonomous vehicle approaches a pedestrian crossing on an unsignalized road section, the motion status information of the autonomous vehicle is obtained through the CAN bus of the autonomous vehicle; images of pedestrians in front of the autonomous vehicle at the pedestrian crossing are obtained through a camera sensor located above the center of the windshield; and the motion status information of pedestrians is obtained through a four-line LiDAR installed on the front bumper of the autonomous vehicle. The motion status information of the autonomous vehicle includes the speed and acceleration of the autonomous vehicle. The motion status information of the pedestrians is the coordinate position, speed, and acceleration of the pedestrians relative to the autonomous vehicle in a coordinate system with the autonomous vehicle as the origin.

[0008] Step S2: Use the Alphapose algorithm to determine the key point coordinates of the pedestrian skeleton based on the pedestrian image, and determine the pedestrian's orientation through the angular relationship between the coordinates of each key point, and convert the coordinates of each key point of the pedestrian skeleton into a feature matrix.

[0009] Step S3: Using the distance between the pedestrian and the autonomous vehicle, the feature matrix of the pedestrian skeleton, and the pedestrian's orientation as input, the GRU-Transformer network algorithm is used to obtain the predicted pedestrian crossing intention;

[0010] Step S4: Model the autonomous vehicle's passage decision as a Markov decision process, define the state space and action space, and initialize the reward function and action value function of the inverse reinforcement learning model;

[0011] Step S5: Fit the reward function using a deep neural network. The input is the current state s and the action a, and the output is the reward function value.

[0012] Step S6: Train the inverse reinforcement learning model, generate a state-action sequence using the current reward function and policy, compare it with the real historical trajectory dataset of autonomous vehicles, calculate the error and backpropagate to update the parameters of the deep neural network, evaluate and adjust the traffic policy, and output the real-time acceleration of the autonomous vehicle.

[0013] Further, in step S2, the Alphapose algorithm includes a feature extraction network and a pose estimation network. The feature extraction network uses a ResNet-101 network to extract features from the pedestrian image to generate a feature map. The pose estimation network inputs the feature map into a multi-stage two-branch convolutional neural network. Both branches consist of a series of convolutional layers and pooling layers to predict the spatial location and heatmap of the pedestrian skeleton key points. At each stage, the pose estimation network first fuses the feature maps and calculates the loss, and then concatenates the prediction results of the two-branch convolutional neural networks with the feature map to continue training in the next stage, finally obtaining the coordinates of the key points of the pedestrian skeleton.

[0014] In step S2, the method for determining the direction of a pedestrian includes determining that the pedestrian is facing the camera when the Alphapose algorithm detects key points such as the face, shoulders, hands, elbows, knees and feet of the pedestrian skeleton; and determining that the pedestrian is facing away from the camera when the Alphapose algorithm detects key points such as the back, shoulders, hands, elbows, knees and feet of the pedestrian skeleton.

[0015] Further, in step S2, the coordinates of each key point of the pedestrian skeleton are converted into a feature matrix, including,

[0016] Step S21: Determine the skeletal features of the pedestrian, including the bone length of the i-th joint and its adjacent joints at time t. The cosine of the angle between the i-th bone and the horizontal direction at time t The length of the bone from the head to the i-th joint at time t The cosine of the angle between the line connecting the head to the i-th joint at time t and the horizontal direction. Where t = 1, 2, 3, ..., w, w is the total number of time points for pedestrian detection, and i = 1, 2, 3, ..., 17;

[0017] Step S22: Determine the bone length characteristics of the i joints and their adjacent joints at time t. Length eigenvector F γ t and the total length eigenvector F γ ,in: Among them, (x i ,y i ), (x j ,y j Let be the coordinates of the i-th and j-th joints of the pedestrian;

[0018] Bone angle features between the i-th and j-th joints at time t Skeletal angle feature vector F λ t and the total skeletal angle feature vector F λ They are: Length characteristics from the head joint to the i-th joint at time t Length eigenvector F ζ t and the total length eigenvector F ζ They are:

[0019] Angular features of the head joint to the i-th joint at time t Angular eigenvector F κ t The total angular eigenvector F κ They are: The total characteristic matrix F is composed of the above four characteristic matrices F λ F γ , and F κ composition;

[0020] Further, in step S3, the GRU-Transformer network for predicting pedestrian crossing intentions includes a GRU layer for capturing the temporal dependencies in the time series of pedestrian feature information and the time series of autonomous vehicle feature information, a positional order hidden state vector and a positional encoding vector for representing the feature sequence, an encoder for converting the input data into a feature representation, and a prediction layer.

[0021] The prediction layer includes a global average pooling layer and a fully connected layer. The global average pooling layer is used to reduce the number of parameters that the fully connected layer needs to optimize, thereby mitigating the overfitting problem. The fully connected layer is used to transform the input data into a prediction result through weighted operations and nonlinear transformations to obtain the pedestrian's intention to cross the street.

[0022] Furthermore, in step S3, the process of the pedestrian crossing intention prediction model based on the GRU-Transformer network is as follows:

[0023] Step S31: Use the Alphapose model to process pedestrian crossing video to estimate the coordinates of pedestrian skeleton key points and generate pedestrian skeleton features; based on the obtained pedestrian and autonomous vehicle motion state information, construct pedestrian-vehicle distance features, and obtain a pedestrian crossing intention prediction dataset after data standardization and time serialization.

[0024] Step S32: The pedestrian crossing intention prediction dataset includes 3660 samples, of which 1990 are crossing samples and 1670 are not crossing samples. The training set and test set are divided according to the 5-fold cross-validation method. The dataset is randomly shuffled and divided into 5 equal-sized subsets. Each subset contains 398 crossing samples and 334 not crossing samples. The cross-validation cycle is repeated 5 times. Each time, one subset is used as the test set and the other 4 subsets are used as the training set.

[0025] Step S33: Train the pedestrian crossing intention prediction model using the GRU-Transformer network; first, use the training set for forward propagation training to obtain the pedestrian crossing prediction probability, then calculate the loss function, perform backpropagation training, and use the Adam optimizer to update the network weights to minimize the loss function.

[0026] Step S34: Use the test set to evaluate the model's generalization ability and record the test results; if the generalization performance is better than the previous test, save the current model and continue training; otherwise, continue the above process until the preset number of training iterations is reached.

[0027] Furthermore, in step S4, the autonomous vehicle's traffic decision is modeled as a Markov decision process, employing... A six-tuple describes a Markov decision process, where S = {s1, s2, ..., s...} n Let s be the set of n states of the environment in which the autonomous vehicle is located. n Representing the nth state, A = {a1, a2, ..., a...} n Let} be the set of n actions for an autonomous vehicle, and a n Represents the nth action. It is a set of state transition functions. S→A is the set of reward functions, η(0,1) is the discount factor, and D is the distribution of the initial state.

[0028] Furthermore, in step S4, the state space, action space, and state transition model are specifically set as follows:

[0029] The obtained state information of the i-th pedestrian is defined as: s p ={p i ,d i} where p i Let d represent the crossing intention of the i-th pedestrian. i Let be the longitudinal distance between the i-th pedestrian and the autonomous vehicle;

[0030] The state information of an autonomous vehicle is defined as: s e ={v e}, where vg is the speed state space of the autonomous vehicle: s = s p +s e ;

[0031] The action space of an autonomous vehicle is defined as: a = {a y}, where a y It is longitudinal acceleration;

[0032] Let the autonomous vehicle be in state s at time t. t ={p i (t),di (t),v e (t)} take action a t , a t =a y (t), the state obtained at the next time step is denoted as s. t+1 ={p i (t+1),d i (t+1),v e (t+1)}, using the state transition equation to represent state s t to s t+1 The state transition process can be expressed by the following equation:

[0033]

[0034] Furthermore, in step S4, the reward function and the value function are defined as follows:

[0035] Reward Function Each state feature f s A linear mapping to the state reward value representing the utility required to access that state, the reward function. The linear combination of feature f(s,a) and weight vector χ:

[0036] Action value function Q under policy π π (s,a) is Where η is the discount factor;

[0037] The value function update process is as follows: Where α is the learning rate, α∈[0,1], and s′ is the next state obtained by performing action a in state s.

[0038] Further, in step S5, a deep neural network is used to fit a reward function. Based on the collected historical trajectory data of the autonomous vehicle and the predicted pedestrian crossing intentions, the state values ​​and action values ​​of the autonomous vehicle in the human-vehicle interaction event are extracted. The state values ​​are the longitudinal distance between the autonomous vehicle and the pedestrian, the pedestrian's crossing intention, and the speed of the autonomous vehicle. The action values ​​are the longitudinal acceleration of the autonomous vehicle. The input of the deep neural network is the state vector s of the autonomous vehicle at time t. t and action vector a t The output is the reward function value at the current moment.

[0039] Furthermore, in step S6, the training process for the autonomous vehicle traffic strategy is as follows:

[0040] Initialize all weights and bias parameters θ in the deep neural network, and obtain the reward function value through forward propagation;

[0041] The policy function π(s,a) under the current reward is obtained by using the value iteration method after iteration of the inverse reinforcement learning neural network.

[0042] Among them, V π (s t Let s be the state under policy π. t The value represents the state s t Begin following the expected discount cumulative reward of policy π; the policy function is calculated as follows:

[0043] π(s,a)=exp(Q π (s,a)-V π (s t Among them, Q π (s,a) represents the long-term reward obtained by performing action a in state s, and η is the discount factor;

[0044] Generate M sets of state-action sequences D using the current policy π. π ={(s1,a1),(s2,a2)…,(s M ,a M N sets of state-action sequences D are obtained from the real-world historical trajectory dataset of autonomous vehicles collected from human-vehicle interaction scenarios at pedestrian crossings on the road. E ={(s1,a1),(s2,a2)…,(s N ,a N Let p be the joint distribution of the state and action of the real autonomous vehicle and the policy π. E (s,a) and p π (s,a):

[0045] p E (s,a)=P(s,a|D E ), p π (s,a)=P(s,a|D π The goal of inverse reinforcement learning is to make the agent's policy distribution p π (s,a) represents a more realistic distribution of behavior in autonomous vehicles. E (s,a), therefore the loss function L(π) is defined as the KL divergence between the two distributions:

[0046]

[0047] Deep neural networks are trained by minimizing the loss function through gradient updates.

[0048] θ=θ+α*g

[0049] Where g is the gradient of the loss function, and α is the learning efficiency of the inverse reinforcement learning model; training ends when the gradient g is less than a preset threshold, resulting in the final reward function model and generating the final driver-like traffic strategy π for autonomous vehicles. * Outputs the real-time acceleration of autonomous vehicles;

[0050] Compared with existing technologies, the beneficial effects of this invention are that the autonomous vehicle is a driverless autonomous vehicle. When the autonomous vehicle approaches an intersection where there is no traffic signal guidance at a pedestrian crossing, it makes a passage decision based on the acquired motion state information of the autonomous vehicle and the motion state information of the pedestrian. It then adjusts the passage strategy by comparing it with the real historical trajectory dataset of the autonomous vehicle collected from the human-vehicle interaction scenario at the pedestrian crossing, thereby further improving the safety of the passage strategy. Determining the driving strategy of the autonomous vehicle based on the specific driving situation effectively improves the safety, efficiency, and comfort of the driverless autonomous vehicle.

[0051] Furthermore, in designing the Transformer network, the attention mechanism allocates weights differently for different processing regions, giving more attention to important content and less attention to secondary or irrelevant content, which effectively improves the efficiency of information extraction.

[0052] Furthermore, the GRU layer can compress the temporal dimension information in pedestrian and autonomous vehicle feature information, effectively simplifying multiple features for pedestrian intention prediction; the hidden state vector and position encoding vector are used to represent the positional order of the feature sequence; the encoder is used to convert the input data into feature representation; the prediction layer includes a global average pooling layer and a fully connected layer, where the global average pooling layer is used to reduce the number of parameters that the fully connected layer needs to optimize, thereby mitigating the overfitting problem, and the fully connected layer is used to predict pedestrian crossing intentions, effectively improving the safety of driverless autonomous vehicles. Attached Figure Description

[0053] Figure 1 This is a flowchart illustrating the steps of an autonomous vehicle traffic decision-making method for urban unsignaled road sections according to an embodiment of the present invention.

[0054] Figure 2 This is a schematic diagram of human skeletal features according to an embodiment of the present invention;

[0055] Figure 3 This is a diagram of the GRU-Transformer network structure according to an embodiment of the present invention;

[0056] Figure 4 This is a diagram of a deep neural network structure according to an embodiment of the present invention. Detailed Implementation

[0057] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0058] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0059] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0060] Furthermore, it should be noted that, in the description of the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0061] Please see Figure 1 , Figure 2 , Figure 3 as well as Figure 4 The diagrams shown are, respectively, a flowchart of the steps, a schematic diagram of human skeletal features, a GRU-Transformer network structure diagram, and a deep neural network structure diagram for an autonomous vehicle traffic decision-making method for urban unsignaled road sections according to an embodiment of the present invention; the embodiment of the present invention provides an autonomous vehicle traffic decision-making method for urban unsignaled road sections, comprising:

[0062] Step 1: Collect vehicle motion status information

[0063] When an autonomous vehicle approaches a pedestrian crossing on an unsignalized road, the system acquires the vehicle's motion status information via its CAN bus, obtains images of pedestrians in front of the vehicle at the pedestrian crossing via a camera sensor positioned above the center of the windshield, and acquires the pedestrian's motion status information via a four-line lidar sensor mounted on the vehicle's front bumper. The autonomous vehicle's motion status information includes its speed and acceleration. The pedestrian's motion status information includes its position relative to the autonomous vehicle, its speed, and its acceleration in a coordinate system with the autonomous vehicle as the origin.

[0064] Step 2: Calculation of key point coordinates of pedestrian skeleton

[0065] The Alphapose algorithm is used to estimate the coordinates of key points on a pedestrian's skeleton, and the pedestrian's orientation is determined by the angles between these key points. The key points are then converted into a feature matrix. The Alphapose deep convolutional neural network algorithm consists of two parts: a feature extraction network and a pose estimation network. The feature extraction network uses a ResNet-101 network to extract features from the input image and generate feature maps. The pose estimation network inputs the feature maps into a multi-stage, two-branch convolutional neural network. Both branches consist of a series of convolutional and pooling layers to predict the spatial location and heatmap of the pedestrian's skeletal key points. At each stage, the network first fuses the feature maps and calculates the loss, then concatenates the predictions from the two branches with the original input, and continues training to the next stage, ultimately obtaining the coordinates of the pedestrian's skeletal key points.

[0066] The four skeletal features of a pedestrian include: (1) the skeletal length of the i-th joint and its adjacent joints at time t. (2) The cosine of the angle between the i-th bone and the horizontal direction at time t. (3) The length of the bone from the head to the i-th joint at time t. (4) The cosine of the angle between the line connecting the head to the i-th joint at time t and the horizontal direction.

[0067] The length feature of the i-th bone at time t Length eigenvector F γ t and the total length eigenvector F γ They are:

[0068]

[0069]

[0070]

[0071] Among them, (xi ,y i ), (x j ,y j ) represents the coordinates of the i-th and j-th pedestrian skeleton key points.

[0072] The angle feature of the i-th bone at time t Skeletal angle feature vector F λ t and the total skeletal angle feature vector F λ They are:

[0073]

[0074]

[0075]

[0076] Length characteristics from the head joint to the i-th joint at time t Length feature vector and total length eigenvector They are:

[0077]

[0078]

[0079]

[0080] Angular features of the head joint to the i-th joint at time t Angular eigenvector F κ t The total angular eigenvector F κ They are:

[0081]

[0082]

[0083]

[0084] The characteristic matrix F is composed of the four characteristic matrices F mentioned above. λ F γ F ζ and F κ composition.

[0085]

[0086] Step 3: Pedestrian Crossing Intent Prediction Model Based on GRU-Transformer

[0087] The GRU-Transformer network consists of a GRU layer, hidden state vectors, position encoding vectors, an encoder, and a prediction layer.

[0088] Sub-step 1: Use a lightweight Alphapose model to process pedestrian crossing video to estimate the coordinates of key points of pedestrian skeletons and generate pedestrian skeleton features; construct pedestrian-vehicle distance features based on the human and vehicle motion state information obtained by the vehicle-mounted environmental perception sensor, and obtain a pedestrian crossing intention prediction dataset after data standardization and time serialization.

[0089] To reduce the impact of numerical differences between features, feature standardization is employed.

[0090] x′ ij =(x ij -μ j ) / σ j

[0091] Where, x ij μ is the j-th feature of the i-th sample; j σ is the mean of the j-th feature; j Let be the standard deviation of the j-th feature. To combine current and historical data to predict pedestrian crossing intentions, the Keras framework's interface is used to serialize the samples over time.

[0092] Sub-step 2: GRU network design

[0093] The GRU layer can compress the temporal dimension information in pedestrian and autonomous vehicle feature information, effectively simplifying multiple features for pedestrian intention prediction; the hidden state vector and the position encoding vector are used to represent the positional order of the feature sequence; the encoder is used to convert the input data into feature representation; the prediction layer includes a global average pooling layer and a fully connected layer, where the global average pooling layer is used to reduce the number of parameters that the fully connected layer needs to optimize in order to alleviate the overfitting problem, and the fully connected layer is used to predict pedestrian crossing intentions.

[0094] GRU uses the gating principle to control the flow of information, through the reset gate. t With Update Gate z t composition.

[0095] Reset door r t and update gate z t The calculation formula is:

[0096] r t =σ(W r x r +U r h t-1 ), z t =σ(W z x r +Uz h t-1 In the formula, σ is the sigmoid activation function, used to prevent gradient vanishing and exploding; h t-1 This is the hidden state from the previous time step; W r W represents the weights of the input layer in the reset gate. z U represents the weights of the input layer in the update gate; r The weight of the hidden layer in the reset gate at the previous moment; U z This represents the weight of the hidden layer in the update gate at the previous time step.

[0097] By resetting the gate r t A new feature hidden state can be obtained. Right now In the formula, W h U represents the weights of the input layer in the tanh activation function; h is the weight matrix of the hidden layer in the tanh activation function at the previous time step; e represents the Hadamard product of the matrices; tanh is the hyperbolic tangent function.

[0098] By updating gate z t The current feature hiding state h can be obtained. t ,Right now

[0099] In the formula, the current feature hiding state h t The result is the feature dimensionality reduction.

[0100] To fully utilize the positional information of the feature sequence itself, a positional encoder composed of sine and cosine functions is used to extract features, i.e.

[0101] P (p,2i) =sin(p / 1000) 2i / d )

[0102] P (p,2i+1) =cos(p / 1000) 2i / d )

[0103] In the formula, p is the current position of the feature; 2i is the even dimension; 2i+1 is the odd dimension; and d is the number of input features.

[0104] Sub-step 3: Transformer Network Design

[0105] Attention mechanisms allocate weights differently to different processing regions, giving more attention to important content and less attention to secondary or irrelevant content, effectively improving information extraction efficiency and being widely used in image processing and other fields. The Transformer model further extracts the relationships between features based on the attention mechanism, and performs feature filtering and fusion, possessing powerful information compression and feature capture capabilities. The encoder used in the Transformer model mainly consists of multi-head attention layers and feedforward network layers based on the attention mechanism. Residual networks and normalization layers are added to each sub-layer to accelerate convergence and prevent network degradation.

[0106] The multi-head attention layer is used to capture important features of the input. It linearly transforms the input matrices Q, K, and V and then feeds them into b scaling dot product units for computation. The results of these b scaling dot product units are concatenated and then linearly mapped to output the final result. This network structure possesses the ability to learn features from different subspaces.

[0107] The scaled dot product unit attention function (ATT) and the multi-head attention layer function (MHA) are respectively

[0108]

[0109] MHA(Q,K,V)=Con cat(b1,b2…,b b, ) W.

[0110] Where, d k is the input feature dimension; softmax is the activation function; W o Let be the weight matrix; Concat is the vector concatenation operation; bi is the feature of the i-th attention layer, where i∈{1,2,…,b'}, and the calculation formula is:

[0111] b i =ATT(QW i Q ,KW i K VW i V )

[0112] Among them, W i Q 、W i K and W i V These are all weight matrices corresponding to the input features of the i-th attention layer.

[0113] Feedforward network layers are used to prevent model output degradation and mainly consist of two linear transformations. The specific calculation formula for the feedforward network function is as follows:

[0114] FFN(x) = max(0, xZ1+c1)Z2+c2

[0115] In the formula: Z1 and Z2 are weight matrices; c1 and c2 are bias terms; x is the input of the feedforward network layer.

[0116] Sub-step 4: The dataset contains 3660 samples, including 1990 crossing samples and 1670 non-crossing samples. A 5-fold cross-validation method is used to divide the dataset into training and test sets. The dataset is randomly shuffled and divided into 5 equal subsets, each containing 398 crossing samples and 334 non-crossing samples. This cross-validation cycle is repeated 5 times, using one subset as the test set and the remaining four subsets as the training set each time.

[0117] Sub-step 5: Train the pedestrian crossing intention prediction model using the GRU-Transformer network. First, perform forward propagation training using the training set to obtain the pedestrian crossing prediction probability. Then, calculate the loss function, perform backpropagation training, and use the Adam optimizer to update the network weights to minimize the loss function.

[0118] Sub-step 6: Evaluate the model's generalization ability using the test set and record the test results. If the generalization performance is better than the previous test, save the current model and continue training; otherwise, continue the above process until the preset number of training iterations is reached.

[0119] Step 4: Modeling Traffic Decisions for Autonomous Vehicles

[0120] The autonomous vehicle's traffic decision-making is modeled as a Markov Decision Process (MDP), and the following approach is adopted. A six-tuple describes an MDP, where S = {s1, s2, ..., s...} n Let s be the set of n states of the environment in which the autonomous vehicle is located. n Representing the nth state, A = {a1, a2, ..., a...} n Let} be the set of n actions for an autonomous vehicle, and a n Represents the nth action. It is a set of state transition functions. Let be the set of reward functions, η∈(0,1) be the discount factor, and D be the distribution of the initial state.

[0121] The state space is a description of the environment in which an autonomous vehicle is located. An autonomous vehicle located at a pedestrian crossing will be affected by pedestrians crossing the pedestrian crossing.

[0122] Considering the pedestrian's intention to cross the street and the relative motion between the pedestrian and the autonomous vehicle, the state information of the i-th pedestrian in the vicinity is defined as: s p ={pi ,d i}

[0123] Where, p i Let d represent the crossing intention of the i-th pedestrian. i Let be the longitudinal distance between the i-th pedestrian and the autonomous vehicle.

[0124] The status information of a vehicle is defined as follows:

[0125] s e ={v e}

[0126] Among them, v e Indicates the vehicle's speed.

[0127] Therefore, the state space can be represented as:

[0128] s = s p +s e

[0129] The action space of an autonomous vehicle is defined as:

[0130] a={a y}

[0131] Among them, a y This is the longitudinal acceleration.

[0132] Let the autonomous vehicle be in state s at time t. t ={p i (t),d i (t),v e (t)} take action a t , a t =a y (t), the state obtained at the next time step is denoted as s. t+1 ={p i (t+1),d i (t+1),v e (t+1)}, using the state transition equation to represent state s t to s t+1 The state transition process can be expressed by the following equation:

[0133]

[0134] Step 5: Define the reward function and value function

[0135] Reward Function Each state feature is linearly mapped to a state reward value representing the utility required to access that state. Assuming a reward function It is feature f sLinear combination with weight vector χ:

[0136]

[0137] Based on the collected trajectory data of autonomous vehicles and the predicted pedestrian crossing intentions, the state and action values ​​of the autonomous vehicle in each vehicle-autonomous interaction event are extracted. The state values ​​are the longitudinal distance between the autonomous vehicle and the pedestrian, the pedestrian's crossing intention, and the speed of the autonomous vehicle. The action value is the longitudinal acceleration of the autonomous vehicle. A deep neural network (DNN) is used to fit the reward function. The input of the DNN is the autonomous vehicle's state vector s at time t. t and action vector a t The output is the reward function value at the current moment.

[0138] Action value function Q under policy π π (s,a) can be represented as

[0139]

[0140] Where η is the discount factor.

[0141] The value function update process can be represented as follows:

[0142]

[0143] Where α is the learning rate, α∈[0,1], and s′ is the next state obtained by performing action a in state s.

[0144] Step Six: Training the Pathway Decision Network

[0145] Sub-step 1: Initialize all weights and bias parameters θ of the deep neural network (DNN).

[0146] Sub-step 2: Obtain a preliminary reward function using the current neural network parameters.

[0147] Sub-step 3: Use the value iteration method to solve for the policy function π(s,a) output by the inverse reinforcement learning neural network after iteration.

[0148]

[0149] Among them, V π (s t Let s be the state under policy π. t The value represents the state s t Start following the expected discount accumulation rewards of strategy π.

[0150] The strategy function is calculated using the following formula:

[0151]

[0152] π(s,a)=exp(Q π (s,a)-V π (s t ))

[0153] Among them, Q π (s,a) represents the long-term reward obtained by performing action a in state s, and η is the discount factor.

[0154] Sub-step 4: Select an action using the current policy π, obtain a new state and reward through environmental feedback, and generate these state-action sequences D. π Store in the buffer.

[0155] D π ={(s1,a1),(s2,a2)…,(s M ,a M )}

[0156] Sub-step 5: Obtain the state-action sequence D from the real autonomous vehicle historical trajectory dataset collected from the human-vehicle interaction scenario at the pedestrian crossing. E .

[0157] D E ={(s1,a1),(s2,a2)…,(s N ,a N )}

[0158] Sub-step 6: Define the joint state-action distribution of the real driver and policy π as p E (s,a) and p π (s,a)

[0159] p E (s,a)=P(s,a|D E )

[0160] p π (s,a)=P(s,a|D π )

[0161] Sub-step 7: The goal of inverse reinforcement learning is to make the agent's policy distribution p π (s,a) represents a more realistic driver behavior distribution p E (s,a), therefore the loss function L(π) is defined as the KL divergence of the two policy distributions. The DNN network is trained by minimizing the loss function through gradient updates.

[0162]

[0163]

[0164] θ+θ+α*g

[0165] Where α represents the learning efficiency of the inverse reinforcement learning model, and g represents the gradient of the loss function.

[0166] Sub-step 8: Iterative solution. Each iteration includes sub-steps 2 to 7. Training ends when the gradient g is less than a preset threshold, yielding the final reward function model and generating the final autonomous vehicle driver access strategy π. * It outputs the real-time acceleration of autonomous vehicles.

[0167]

[0168] Thus far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will fall within the scope of protection of the present invention.

[0169] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that the present invention is susceptible to various modifications and variations. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A method for autonomous vehicle traffic decision-making in urban areas with no traffic signals, characterized in that, include: Step S1: When the autonomous vehicle approaches a pedestrian crossing on an unsignalized road section, the motion status information of the autonomous vehicle is obtained through the CAN bus of the autonomous vehicle; images of pedestrians in front of the autonomous vehicle at the pedestrian crossing are obtained through a camera sensor located above the center of the windshield; and the motion status information of pedestrians is obtained through a four-line LiDAR installed on the front bumper of the autonomous vehicle. The motion status information of the autonomous vehicle includes the speed and acceleration of the autonomous vehicle. The motion status information of the pedestrians is the coordinate position, speed, and acceleration of the pedestrians relative to the autonomous vehicle in a coordinate system with the autonomous vehicle as the origin. Step S2: Use the Alphapose algorithm to determine the key point coordinates of the pedestrian skeleton based on the pedestrian image, and determine the pedestrian's orientation by the angular relationship between the coordinates of each key point, and convert the coordinates of each key point of the pedestrian skeleton into a feature matrix. Step S3: Using the distance between the pedestrian and the autonomous vehicle, the feature matrix of the pedestrian skeleton, and the pedestrian's orientation as input, the GRU-Transformer network algorithm is used to obtain the predicted pedestrian crossing intention; Step S4: Model the autonomous vehicle's passage decision as a Markov decision process, define the state space and action space, and initialize the reward function and action value function of the inverse reinforcement learning model; Step S5: Fit the reward function using a deep neural network, with the input being the autonomous vehicle state vector s at time t. t and action vector a t The output is the reward function value at the corresponding moment. Based on the collected historical trajectory data of the autonomous vehicle and the predicted pedestrian crossing intention, the state value and action value of the autonomous vehicle in the human-vehicle interaction event are extracted. The state value is the longitudinal distance between the autonomous vehicle and the pedestrian, the pedestrian crossing intention and the speed of the autonomous vehicle, and the action value is the longitudinal acceleration of the autonomous vehicle. Step S6: Train the inverse reinforcement learning model, generate a state-action sequence using the current reward function and policy, compare it with the real historical trajectory dataset of autonomous vehicles, calculate the error and backpropagate to update the parameters of the deep neural network, evaluate and adjust the traffic policy, and output the real-time acceleration of the autonomous vehicle.

2. The method for autonomous vehicle traffic decision-making in urban unsignaled road sections according to claim 1, characterized in that, In step S2, the Alphapose algorithm includes a feature extraction network and a pose estimation network. The feature extraction network uses a ResNet-101 network to extract features from the pedestrian image to generate a feature map. The pose estimation network inputs the feature map into a multi-stage two-branch convolutional neural network. Both branches consist of a series of convolutional layers and pooling layers to predict the spatial location and heatmap of the pedestrian skeleton key points. At each stage, the pose estimation network first fuses the feature maps and calculates the loss, and then concatenates the prediction results of the two-branch convolutional neural networks with the feature map to continue training in the next stage, finally obtaining the coordinates of the key points of the pedestrian skeleton. In step S2, the method for determining the direction of the pedestrian includes determining that the pedestrian is facing the camera when the Alphapose algorithm detects key points of the pedestrian's face, shoulders, hands, elbows, knees and feet; and determining that the pedestrian is facing away from the camera when the Alphapose algorithm detects key points of the pedestrian's back, shoulders, hands, elbows, knees and feet.

3. The method for autonomous vehicle traffic decision-making in urban unsignaled road sections according to claim 2, characterized in that, In step S2, the coordinates of each key point of the pedestrian skeleton are converted into a feature matrix. include, Step S21: Determine the skeletal features of the pedestrian, including the bone length of the i-th joint and its adjacent joints at time t. The cosine of the angle between the i-th bone and the horizontal direction at time t The length of the bone from the head to the i-th joint at time t The cosine of the angle between the line connecting the head to the i-th joint at time t and the horizontal direction. Where t = 1, 2, 3, ..., w, w is the total number of time points for pedestrian detection, and i = 1, 2, 3, ..., 17; Step S22: Determine the bone length characteristics of the i joints and their adjacent joints at time t. Length eigenvector F γ t and the total length eigenvector F γ ,in: Among them, (x i ,y i ), (x j ,y j Let be the coordinates of the i-th and j-th joints of the pedestrian; Bone angle features between the i-th and j-th joints at time t Skeletal angle feature vector F λ t and the total skeletal angle feature vector F λ They are: Length characteristics from the head joint to the i-th joint at time t Length feature vector and total length eigenvector They are: Angular features of the head joint to the i-th joint at time t Angular eigenvector F κ t The total angular eigenvector F κ They are: The total characteristic matrix F is composed of the above four characteristic matrices F λ F γ , and F κ composition; 4. The method for autonomous vehicle traffic decision-making in urban unsignaled road sections according to claim 3, characterized in that, In step S3, the GRU-Transformer network for predicting pedestrian crossing intentions includes a GRU layer for capturing the temporal dependencies in the time series of pedestrian feature information and the time series of autonomous vehicle feature information, a positional order hidden state vector and a positional encoding vector for representing the feature sequence, an encoder for converting the input data into a feature representation, and a prediction layer. The prediction layer includes a global average pooling layer and a fully connected layer. The global average pooling layer is used to reduce the number of parameters that the fully connected layer needs to optimize, thereby mitigating the overfitting problem. The fully connected layer is used to transform the input data into a prediction result through weighted operations and nonlinear transformations to obtain the pedestrian's intention to cross the street.

5. The method for autonomous vehicle traffic decision-making in urban unsignaled road sections according to claim 4, characterized in that, In step S3, the process of the pedestrian crossing intention prediction model based on the GRU-Transformer network is as follows: Step S31: Use the Alphapose model to process pedestrian crossing video to estimate the coordinates of pedestrian skeleton key points and generate pedestrian skeleton features; based on the obtained pedestrian and autonomous vehicle motion state information, construct pedestrian-vehicle distance features, and obtain a pedestrian crossing intention prediction dataset after data standardization and time serialization. Step S32: The pedestrian crossing intention prediction dataset includes 3660 samples, of which 1990 are crossing samples and 1670 are not crossing samples. The training set and test set are divided according to the 5-fold cross-validation method. The dataset is randomly shuffled and divided into 5 equal-sized subsets. Each subset contains 398 crossing samples and 334 not crossing samples. The cross-validation cycle is repeated 5 times. Each time, one subset is used as the test set and the other 4 subsets are used as the training set. Step S33: Train the pedestrian crossing intention prediction model using the GRU-Transformer network; first, use the training set for forward propagation training to obtain the pedestrian crossing prediction probability, then calculate the loss function, perform backpropagation training, and use the Adam optimizer to update the network weights to minimize the loss function. Step S34: Use the test set to evaluate the model's generalization ability and record the test results; If the generalization performance is better than the previous test, save the current model and continue training; otherwise, continue the above process until the preset number of training iterations is reached.

6. The method for autonomous vehicle traffic decision-making in urban unsignaled road sections according to claim 5, characterized in that, In step S4, the autonomous vehicle's traffic decision is modeled as a Markov decision process, using... A six-tuple describes a Markov decision process, where S = {s1, s2, ..., s...} n Let s be the set of n states of the environment in which the autonomous vehicle is located. n Representing the nth state, A = {a1, a2, ..., a...} n Let} be the set of n actions for an autonomous vehicle, and a n Represents the nth action. It is a set of state transition functions. Let be the set of reward functions, γ∈(0,1) be the discount factor, and D be the distribution of the initial state.

7. The method for autonomous vehicle traffic decision-making in urban unsignaled road sections according to claim 6, characterized in that, In step S4, the state space, action space, and state transition model are specifically set as follows: The obtained state information of the i-th pedestrian is defined as: s p ={p i ,d i } where p i Let d represent the crossing intention of the i-th pedestrian. i Let be the longitudinal distance between the i-th pedestrian and the autonomous vehicle; The state information of an autonomous vehicle is defined as: s e ={v e }, where v g Let be the speed of the autonomous vehicle, and the state space be: s = s p +s e ; The action space of an autonomous vehicle is defined as: a = {a y }, where a y It is longitudinal acceleration; Let the autonomous vehicle be in state s at time t. t ={p i (t),d i (t),v e (t)} take action a t , a t =a y (t), the state obtained at the next time step is denoted as s. t+1 ={p i (t+1),d i (t+1),v e (t+1)}, using the state transition equation to represent state s t to s t+1 The state transition process is expressed by the following equation:

8. The method for autonomous vehicle traffic decision-making in urban unsignaled road sections according to claim 7, characterized in that, In step S4, the reward function and the value function are defined as follows: Reward Function Each state feature f s A linear mapping to the state reward value representing the utility required to access that state, the reward function. The linear combination of feature f(s,a) and weight vector χ: Action value function Q under policy π π (s,a) is Where η is the discount factor; s represents the state, and a represents the action to be performed.

9. The method for autonomous vehicle traffic decision-making in urban unsignaled road sections according to claim 8, characterized in that, In step S6, the training process for the autonomous vehicle traffic strategy is as follows: Initialize all weights and bias parameters θ in the deep neural network, and obtain the reward function value through forward propagation; The policy function π(s,a) under the current reward is obtained by using the value iteration method after iteration of the inverse reinforcement learning neural network. Among them, V π (s t Let s be the state under policy π. t The value represents the state s t Begin following the expected discount cumulative reward of policy π; the policy function is calculated as follows: π(s,a)=exp(Q π (s,a)-V π (s t Among them, Q π (s,a) represents the long-term reward obtained by performing action a in state s, and η is the discount factor; Generate M sets of state-action sequences D using the current policy π. π ={(s1,a1),(s2,a2)…,(s M ,a M N sets of state-action sequences D are obtained from the real-world historical trajectory dataset of autonomous vehicles collected from human-vehicle interaction scenarios at pedestrian crossings on the road. E ={(s1,a1),(s2,a2)…,(s N ,a N Let p be the joint distribution of the state and action of the real autonomous vehicle and the policy π. E (s,a) and p π (s,a): p E (s,a)=P(s,a|D E ), p π (s,a)=P(s,a|D π The goal of inverse reinforcement learning is to make the agent's policy distribution p π (s,a) represents a more realistic distribution of behavior in autonomous vehicles. E (s,a), therefore the loss function L(π) is defined as the KL divergence between the two distributions: Deep neural networks are trained by minimizing the loss function through gradient updates. Where g is the gradient of the loss function, and α is the learning efficiency of the inverse reinforcement learning model; training ends when the gradient g is less than a preset threshold, resulting in the final reward function model and generating the final driver-like traffic strategy π for autonomous vehicles. * Outputs the real-time acceleration of autonomous vehicles;

Images